-- Logs begin at Wed 2022-06-01 03:05:40 UTC, end at Wed 2022-06-01 04:53:15 UTC. -- Jun 01 04:06:13.964516 dv-cinder-667108-37-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 01 04:06:13.973184 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 1 04:06:13 2022] *** Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: nodename: dv-cinder-667108-37-iscsi Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: machine: x86_64 Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: clock source: unix Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: pcre jit disabled Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: detected number of CPU cores: 4 Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: current working directory: / Jun 01 04:06:13.975216 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: detected binary path: /usr/bin/uwsgi-core Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: your processes number limit is 47680 Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: your memory page size is 4096 bytes Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: detected max file descriptor number: 2048 Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: lock engine: pthread robust mutexes Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: thunder lock: enabled Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 01 04:06:13.975938 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 01 04:06:14.020849 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: Python main interpreter initialized at 0x55cc1fa019c0 Jun 01 04:06:14.020849 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: python threads support enabled Jun 01 04:06:14.020849 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: your server socket listen backlog is limited to 100 connections Jun 01 04:06:14.020849 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: your mercy for graceful operations on workers is 90 seconds Jun 01 04:06:14.021175 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: mapped 403077 bytes (393 KB) for 2 cores Jun 01 04:06:14.021250 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: *** Operational MODE: preforking *** Jun 01 04:06:14.021524 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: *** uWSGI is running in multiple interpreter mode *** Jun 01 04:06:14.021524 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: spawned uWSGI master process (pid: 80698) Jun 01 04:06:14.021720 dv-cinder-667108-37-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 01 04:06:14.022679 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: spawned uWSGI worker 1 (pid: 80710, cores: 1) Jun 01 04:06:14.022679 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: spawned uWSGI worker 2 (pid: 80711, cores: 1) Jun 01 04:06:14.024115 dv-cinder-667108-37-iscsi devstack@keystone.service[80698]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 01 04:06:19.750923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 01 04:06:19.750923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: from cryptography.utils import int_from_bytes Jun 01 04:06:19.751711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 01 04:06:19.751711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: from cryptography.utils import int_from_bytes Jun 01 04:06:19.751815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 01 04:06:19.751815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: from cryptography.utils import int_from_bytes Jun 01 04:06:19.752217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 01 04:06:19.752217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: from cryptography.utils import int_from_bytes Jun 01 04:06:19.802646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 01 04:06:19.802838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 01 04:06:19.803023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] command line args: [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 01 04:06:19.803316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 01 04:06:19.803511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ================================================================================ {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 01 04:06:19.803748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 01 04:06:19.803838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] admin_token = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.803950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 01 04:06:19.804030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] config_dir = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.804149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] command line args: [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 01 04:06:19.804278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.804374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] config_source = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.804456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 01 04:06:19.804534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] debug = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.804611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ================================================================================ {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 01 04:06:19.804708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.804967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] admin_token = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] config_dir = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] config_source = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] debug = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.805970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_config_append = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_dir = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_file = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.806956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_config_append = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_dir = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_file = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.807943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.808953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.809920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.810058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.810236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.810408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.810584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.810764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.810933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.811106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] publish_errors = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.811106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.811477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.811648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.811648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.811957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] publish_errors = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] standard_threads = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.812998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.813463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.813578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.813735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.813852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] standard_threads = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.813965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.813965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] use_journal = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] use_json = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] use_stderr = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] use_syslog = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.814917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] use_journal = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.815029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.815150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] use_json = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.815150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] use_stderr = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.815391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.815513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] use_syslog = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.815628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.815781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 01 04:06:19.815911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.816940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.817985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818467 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.818920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.819904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.820948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.821903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.822903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.823966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.824937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.825887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826001 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.826933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.827993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.828910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.829954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.830971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.831987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.832924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.833996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.834933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.835903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.836948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.837892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.838956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.839908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.external = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.external = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.840958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.password = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.password = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] auth.token = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] auth.token = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.841936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.842895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.843991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.844890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.845959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.846903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.847018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.847687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.847806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.847922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.848973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.849907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.850954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.851953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.852998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.853940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.854984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.855936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.856992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.857994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.858922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.859999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.860925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.861980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.862095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.862209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.862323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.862446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.862565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.862697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.863373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.863601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.863718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.863847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.863965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.864892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.865949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.866897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.867998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.868295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.868409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.868523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.868644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.868760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.868886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.869939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.870989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.871942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.872921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.873977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] role.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] role.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.874905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] role.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] role.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.875906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.876950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.877999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.878979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.879930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.880967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.881888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882001 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.882936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.883052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.883438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.883556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.883670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.883786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.883913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.884957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.885987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.886920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.connection = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.887997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.connection = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.888999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.889933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.890974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80710) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 01 04:06:19.891852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 01 04:06:19.891852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 01 04:06:19.917389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.917603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.917827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.918060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.918364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.918530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.926906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.927654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.955399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.955630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.955770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:19.956006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.010898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.012885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.061221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.061452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.061537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.061762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.127737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.127905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.128990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.129210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.129449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.153919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154229 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.154676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.166461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.166786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.178901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.179987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.180069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.180199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80710) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.180281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.180415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.180634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 01 04:06:20.197220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.197431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 01 04:06:20.197693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.197777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.197911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 01 04:06:20.198008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 01 04:06:20.198212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.198292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.198437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 01 04:06:20.198518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.198709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.198792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.198960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.199043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.199218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.199601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.210448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.210725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.210995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.211077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.211274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.211389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.211633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.211916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.219322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.220344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.223176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.223483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.223766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.224044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.224231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.224312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.224514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.224676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.224797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.224948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.225079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.225220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.225363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.225487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.225738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.225819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.226013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.226095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.226292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.226566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.226845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.227114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.252347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.253500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.254097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.254377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.255222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.255531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.259959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.260243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.260519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.261043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.261327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.261613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.267800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.268081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.269721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.270006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.273264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.273661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.273944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.274228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.274506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.274669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.274775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.274907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.275042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.275196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.275334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.275510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.275793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.276075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.276352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.276629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.295183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.295483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.295920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.296204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.300077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.300923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.304026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.304327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.304623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.304922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.309265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.309938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.312844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.313598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.318406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.318707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.318993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.319282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.319550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.319633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.319853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.319946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.320140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.320426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.320708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.320993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.340713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.340828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.340996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.341097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.341265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.341349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.341605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.341765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.349985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.350902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.351752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.352035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.352658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.352934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.355082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.355399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.355706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.355978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.356062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.356255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.356351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.356533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.356830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.357121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.369650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.369936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.370219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.370430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.370512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.370716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.370806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.371000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.371117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.371281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.371403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.371589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.371873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.372148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.390320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.390787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.393549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.394647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.397479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.397981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.399096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.399606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.400792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.401076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.401251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.401359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.401537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.401818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.408812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.409177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.410470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.410822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.413725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.414852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.423911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.424003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.424195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.424279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.424574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.424655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.432439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80710) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.432654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 01 04:06:20.432737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.432935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.433858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.434973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.435976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80710) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.436058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.436237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 01 04:06:20.474390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cc1fa019c0 pid: 80710 (default app) Jun 01 04:06:20.475040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cc1fa019c0 pid: 80711 (default app) Jun 01 04:06:20.477655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f8f29-5311-4445-a4ee-a3c16493ef2d None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:20.477947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f8f29-5311-4445-a4ee-a3c16493ef2d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:20.478224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f8f29-5311-4445-a4ee-a3c16493ef2d None None] PATH_INFO: `/v3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:20.482437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Jun 1 04:06:20 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:29.446942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:29.447539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:29.447539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:29.451942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:06:29.457892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:29.458323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:29.458449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:29.527233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: INFO dbcounter [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] Registered counter for database keystone Jun 01 04:06:29.548217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG oslo_db.sqlalchemy.engines [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80710) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 01 04:06:29.554325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writer thread running {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 01 04:06:29.833619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG passlib.handlers.bcrypt [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80710) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 01 04:06:29.834044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG passlib.handlers.bcrypt [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80710) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 01 04:06:29.889170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:29.890759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:30.012694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 556 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 01 04:06:30.188955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:30.259223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: INFO dbcounter [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] Registered counter for database keystone Jun 01 04:06:30.279483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG oslo_db.sqlalchemy.engines [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80711) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 01 04:06:30.285724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writer thread running {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 01 04:06:30.509407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b4c3-fdb9-48b3-9bf9-e1e7d3809e6f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:30.513606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:30.514259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:30.514550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:30.514833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:30.522814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:30.574500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 01 04:06:30.574845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b560940-fb51-4c91-9000-4af809d41018 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:30.584147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Jun 1 04:06:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 01 04:06:30.590129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:30.604313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446d59-eb84-45a4-a0c5-017cfa5fa3ad None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:30.608736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:30.609850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:30.610140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:30.610421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] PATH_INFO: `/v3/projects/efd554ba71f4463abb4c78dc3cafe52a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:30.611974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] RBAC: Authorizing `identity:get_project(project_id=efd554ba71f4463abb4c78dc3cafe52a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:30.663981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING keystone.common.rbac_enforcer.enforcer [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 01 04:06:30.664329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadda4cd-1e20-473a-8a97-66a32624672a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:30.668981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Jun 1 04:06:30 2022] GET /identity/v3/projects/efd554ba71f4463abb4c78dc3cafe52a => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:32.868373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53d818-09cc-4c32-86ee-c8e52c11f2a1 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:32.868878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53d818-09cc-4c32-86ee-c8e52c11f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:32.868983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53d818-09cc-4c32-86ee-c8e52c11f2a1 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:32.873090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:32.878356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:32.878739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:32.879521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:32.898213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:32.899764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:32.908196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 01 04:06:32.931102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53d818-09cc-4c32-86ee-c8e52c11f2a1 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:32.943959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53d818-09cc-4c32-86ee-c8e52c11f2a1 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:32.948156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c2aa-6921-414e-97a1-29fa01f93e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:32.948790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c2aa-6921-414e-97a1-29fa01f93e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:32.949079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c2aa-6921-414e-97a1-29fa01f93e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:32.949357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c2aa-6921-414e-97a1-29fa01f93e1a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:32.957016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c2aa-6921-414e-97a1-29fa01f93e1a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:32.958786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c2aa-6921-414e-97a1-29fa01f93e1a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:32.967376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Jun 1 04:06:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 01 04:06:32.972589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:32.981426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0a396-3e91-4f99-b434-e1a68ccc64d3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:32.985765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71c26e-4424-4cb6-bd00-6ff7ec8969e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:32.986618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71c26e-4424-4cb6-bd00-6ff7ec8969e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:32.986904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71c26e-4424-4cb6-bd00-6ff7ec8969e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:32.987183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71c26e-4424-4cb6-bd00-6ff7ec8969e7 admin admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:32.989125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71c26e-4424-4cb6-bd00-6ff7ec8969e7 admin admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:32.990954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71c26e-4424-4cb6-bd00-6ff7ec8969e7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:32.992531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Jun 1 04:06:32 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:35.270759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb180d-dd6f-4b6a-9eb0-14551e753d16 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.271280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb180d-dd6f-4b6a-9eb0-14551e753d16 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.271414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb180d-dd6f-4b6a-9eb0-14551e753d16 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.272687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:35.278059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.278452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.278623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.295830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:35.297003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:35.302677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560a81-cf30-47f4-9499-5fc35b75e842 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.302970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560a81-cf30-47f4-9499-5fc35b75e842 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.303258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78560a81-cf30-47f4-9499-5fc35b75e842 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.304624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:35.309494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.309788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.310070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.351108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG passlib.handlers.bcrypt [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80711) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 01 04:06:35.351570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG passlib.handlers.bcrypt [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80711) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 01 04:06:35.377511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:35.398775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:35.400397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:35.401605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:35.424591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:35.498245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:06:35.506773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074b409-9a85-425c-9021-926d7b17bb0e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:35.521293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:35.540784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:35.541599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.541884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.542166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.554570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:35.565736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:35.567250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:35.568117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:06:35.572898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:06:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:06:35.573769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:35.645279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266680d1-e96d-41b2-9b03-44e13726b9c4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:35.647819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:35.659931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:35.678288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d416caa-1180-4ec1-abb1-77b917d6fb36 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:35.685033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:35.686206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.686500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.686793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.716965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:35.717647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.717934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.718219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.718979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:35.719288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:35.720735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:35.734869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:35.741008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:35.773839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:35.775499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:35.845224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1963c8-94d4-4bb0-bc8f-93f167a1566b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:35.878714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:35.880143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:35.880431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:35.880712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:35.881315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:06:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:06:35.882231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:35.884012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:35.885624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:35 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:35.887069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:35.890923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:35.891053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:35.905378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:35.989935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2731f04-8fd9-4227-9e98-15dc146516f7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:35.992331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b377e-891c-415a-bba9-834fd5ac24cc None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:36.023225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:36.024109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:36.024347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:36.024628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:36.025559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:36.026343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:36.026632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:36.026906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:36.028105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:36.029560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:36.031146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:36.037337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:36.044026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:36.045514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:06:36.066411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:36.067422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:36.068275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:06:36.068275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:06:36.068275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:06:36.068275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:36.068275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:06:36.073403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:36.118050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:36.129381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:36.136902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af288cde-5e0c-4207-83ab-fab138f11a7c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:36.170612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf858288-a8dc-49ab-a8b5-447bcdb52083 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:36.171595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:36.172246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:36.172530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:36.172809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:36.173643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.domain.id=default, user.id=c2efe34f830a400eb5eb8876be2a1da6) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:36.173954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:36.175431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf30f-f8e7-4f5a-8b2c-7b0fefb7543b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:36.183079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Jun 1 04:06:36 2022] GET /identity/v3/role_assignments?user.id=c2efe34f830a400eb5eb8876be2a1da6&scope.domain.id=default&role.id=02d9e6449ea143beb203824112640698 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:36.205497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:36.206178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:36.206463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:36.206740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:36.207593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:36.207901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:36.209360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b89013-7aa6-44f4-8f81-e67ba1ef5abf None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:36.229545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:38.931096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f25880-c679-4407-9a63-42b55c1dcac1 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:38.931966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f25880-c679-4407-9a63-42b55c1dcac1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:38.932407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f25880-c679-4407-9a63-42b55c1dcac1 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:38.933910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:38.939640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c1718-29f2-4637-9907-f8a33d596d78 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:38.940036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c1718-29f2-4637-9907-f8a33d596d78 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:38.940214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c1718-29f2-4637-9907-f8a33d596d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:38.967442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c43c1718-29f2-4637-9907-f8a33d596d78 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:38.968855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c43c1718-29f2-4637-9907-f8a33d596d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:39.057743 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:39.100831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f25880-c679-4407-9a63-42b55c1dcac1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:39.214931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d328f52-f919-4ebf-92d0-4556c0549f3a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.215593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d328f52-f919-4ebf-92d0-4556c0549f3a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.216088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d328f52-f919-4ebf-92d0-4556c0549f3a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.217652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 12/24] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:06:39.223876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d30e-030f-42c7-960b-cd443cf0f56c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.224480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d30e-030f-42c7-960b-cd443cf0f56c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.224991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d30e-030f-42c7-960b-cd443cf0f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.249108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-46e1d30e-030f-42c7-960b-cd443cf0f56c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:39.251355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-46e1d30e-030f-42c7-960b-cd443cf0f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:39.264904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f25880-c679-4407-9a63-42b55c1dcac1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:39.320815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:39.321627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.321921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.322204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.342013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:39.349766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:39.360189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:39.365366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:39.369639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 13/25] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:06:39.370761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d659ca8-f398-45f3-a096-21907ce69c41 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.371064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d659ca8-f398-45f3-a096-21907ce69c41 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.379388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d659ca8-f398-45f3-a096-21907ce69c41 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.381109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 14/26] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:39.386585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.386885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.387172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:06:39.388788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:39.389666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:39.390458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:06:39.390458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:06:39.390458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:06:39.390458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:06:39.390458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 13/27] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:39.396135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:39.416518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:39.417752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:39.492668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 15/28] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:06:39.493449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:39.606860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ae544-92e3-469e-b6af-cb8b1b179192 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:39.614764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:39.632067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaba9d8-85a6-4da0-bf8e-253c345a525a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:39.643369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:39.644716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.645103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.645386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.657673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:39.667974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:39.668858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:39.669602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:39.670180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 16/29] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:39.670741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:39.684159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:39.684832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.685121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.685399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.686136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:39.686445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:39.687897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:39.707255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 14/30] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:39.708148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:39.787055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:39.796914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9195776-6aec-423c-ab45-e4d06852713b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:39.839735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fddcf7-a8ee-4983-ad2c-b20af9f61e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:39.840816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddcf7-a8ee-4983-ad2c-b20af9f61e5d None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.841323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddcf7-a8ee-4983-ad2c-b20af9f61e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.841801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddcf7-a8ee-4983-ad2c-b20af9f61e5d None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.842805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fddcf7-a8ee-4983-ad2c-b20af9f61e5d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:39.844448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fddcf7-a8ee-4983-ad2c-b20af9f61e5d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:39.846088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fde8e-9614-47ae-aca7-621480f42c48 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:39.887507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:39.888215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.888505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.888791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.889552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:39.889865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:39.891275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc111d8e-18f7-4b63-a36f-db57daae3477 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:39.901311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 15/31] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:39.902512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767cabb-2081-4f77-b41a-ad40a58156ac None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:39.902813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767cabb-2081-4f77-b41a-ad40a58156ac None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:39.903095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767cabb-2081-4f77-b41a-ad40a58156ac None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:39.904578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 16/32] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:39.905123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767cabb-2081-4f77-b41a-ad40a58156ac None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.022641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767cabb-2081-4f77-b41a-ad40a58156ac None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:40.041396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:40.058218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2a2bb-253c-46c4-8c23-ad317bb4e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:40.058890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a2bb-253c-46c4-8c23-ad317bb4e597 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.059178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a2bb-253c-46c4-8c23-ad317bb4e597 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.059494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a2bb-253c-46c4-8c23-ad317bb4e597 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.060214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2a2bb-253c-46c4-8c23-ad317bb4e597 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:40.061810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2a2bb-253c-46c4-8c23-ad317bb4e597 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:40.109644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 17/33] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jun 1 04:06:39 2022] POST /identity/v3/roles => generated 225 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:40.111814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.112314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.112795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.135923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:40.138027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:40.166013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:40.239774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 18/34] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:40.240888 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.329345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 17/35] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:06:39 2022] POST /identity/v3/projects => generated 301 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:40.330849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77b68a-8fe2-4121-9b48-79667f8e96a0 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.331155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77b68a-8fe2-4121-9b48-79667f8e96a0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.331469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77b68a-8fe2-4121-9b48-79667f8e96a0 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.333085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 18/36] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:40.333716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77b68a-8fe2-4121-9b48-79667f8e96a0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.368214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c0f01-8487-40d0-b1f9-d659345d6254 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:40.413513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b302db-4ec6-43e4-b80b-2d34978f2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:40.414611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b302db-4ec6-43e4-b80b-2d34978f2690 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.414903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b302db-4ec6-43e4-b80b-2d34978f2690 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.415744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b302db-4ec6-43e4-b80b-2d34978f2690 None admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.418100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b302db-4ec6-43e4-b80b-2d34978f2690 None admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:40.420486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b302db-4ec6-43e4-b80b-2d34978f2690 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:40.428207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 19/37] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:40 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:40.428207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c7d0f-3593-49ae-99d9-5e3531d0ab8b None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.428207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c7d0f-3593-49ae-99d9-5e3531d0ab8b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.428207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c7d0f-3593-49ae-99d9-5e3531d0ab8b None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.428576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 20/38] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:40.429739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.430217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.430513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.449271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:40.450518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:40.471502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:40.476116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77b68a-8fe2-4121-9b48-79667f8e96a0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:40.518123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95a2b-f405-4ea3-99f0-d30e16aba77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:40.519555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95a2b-f405-4ea3-99f0-d30e16aba77a None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.519859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95a2b-f405-4ea3-99f0-d30e16aba77a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.520422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95a2b-f405-4ea3-99f0-d30e16aba77a None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.521223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95a2b-f405-4ea3-99f0-d30e16aba77a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:40.523786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95a2b-f405-4ea3-99f0-d30e16aba77a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:40.533161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 21/39] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:40.533161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.629187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 19/40] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jun 1 04:06:40 2022] POST /identity/v3/roles => generated 219 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:40.630695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.630990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.631467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.638645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1bf57-cc05-4bf2-ac4a-e4b2ba51585f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:40.649774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:40.651758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:40.677800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:40.679388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.679686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.680158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.681477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:40.683032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:40.683158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:40.684758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 22/41] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:40.688328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.721107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:40.732816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 20/42] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:40.733688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.814068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8540886-f4e1-4d91-812f-c4ae6b752d18 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:40.846016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfa510-634e-4472-ab41-12dac467a188 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:40.853701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:40.854376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.854665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.854951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.856279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:40.857689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:40.884749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:40.885584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:40.885878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:40.886160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] PATH_INFO: `/v3/domains/default/users/c2efe34f830a400eb5eb8876be2a1da6/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:40.888220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c2efe34f830a400eb5eb8876be2a1da6, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:40.889698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:40.925012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 23/43] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jun 1 04:06:40 2022] POST /identity/v3/roles => generated 223 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:40.925895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:40.948196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 21/44] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Jun 1 04:06:40 2022] PUT /identity/v3/domains/default/users/c2efe34f830a400eb5eb8876be2a1da6/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:06:41.024416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:41.046530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acd4ad-1ab4-402d-89a4-9a908c411936 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:41.087753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4e9cc-c1b5-4193-b8d2-0a09fab4d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:41.088596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4e9cc-c1b5-4193-b8d2-0a09fab4d315 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:41.088891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4e9cc-c1b5-4193-b8d2-0a09fab4d315 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:41.089175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4e9cc-c1b5-4193-b8d2-0a09fab4d315 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:41.090544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4e9cc-c1b5-4193-b8d2-0a09fab4d315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:41.092108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4e9cc-c1b5-4193-b8d2-0a09fab4d315 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:41.094051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 24/45] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:41.100356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:41.126516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:41.213468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e9152-4a6c-4413-9179-0ae036e20dbd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:41.247127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e39cf-b17b-46f4-accf-792318edba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:41.250831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e39cf-b17b-46f4-accf-792318edba60 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:41.250831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e39cf-b17b-46f4-accf-792318edba60 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:41.250831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e39cf-b17b-46f4-accf-792318edba60 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:41.250831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e39cf-b17b-46f4-accf-792318edba60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:41.250831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e39cf-b17b-46f4-accf-792318edba60 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:41.313908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 22/46] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:06:41 2022] POST /identity/v3/projects => generated 312 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:43.208308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35fff-53c4-4f6f-bde7-c7981e1f57d7 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.208855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35fff-53c4-4f6f-bde7-c7981e1f57d7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.208855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35fff-53c4-4f6f-bde7-c7981e1f57d7 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.210288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 25/47] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:06:43.216367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.216886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.216968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.234727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:43.235968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:43.303654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 23/48] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:43.327143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b35fff-53c4-4f6f-bde7-c7981e1f57d7 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.426282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b35fff-53c4-4f6f-bde7-c7981e1f57d7 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:43.459124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185a35d-dcb4-4698-8e44-35e96880652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:43.459884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185a35d-dcb4-4698-8e44-35e96880652b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.460171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185a35d-dcb4-4698-8e44-35e96880652b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.460448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185a35d-dcb4-4698-8e44-35e96880652b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.470241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185a35d-dcb4-4698-8e44-35e96880652b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2185a35d-dcb4-4698-8e44-35e96880652b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:43.477937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:43.478779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:43.479560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 26/49] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:43.484984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.487171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:43.538228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fc5a1-0f35-482f-a3e9-88d742548e52 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.538527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fc5a1-0f35-482f-a3e9-88d742548e52 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.538819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3fc5a1-0f35-482f-a3e9-88d742548e52 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.540217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 27/50] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:06:43.544949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.545240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.545524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.562687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:43.563892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:43.567364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:43.588929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b0c6-b449-41d4-a369-fb88b00608b8 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:43.624464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:43.625158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.625436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.625728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.626491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:43.626808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:43.628238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:43.635994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 28/51] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:06:43.637201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 24/52] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:43.642335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.658839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.719937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:43.740641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54e383-d5e6-4951-85bd-ba3041e3f2a0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:43.757222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad141c-4f65-4705-bca3-01fb7c0793e4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:43.773949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:43.774837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.775127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.775447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.776646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:43.778223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:43.779865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 29/53] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:43 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:43.785630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.791171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:43.792001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.792287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.792566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.793751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:43.795208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:43.796041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:43.796835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 25/54] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:43.802815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.885185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cecfc5-3040-48d1-8649-9d04a26a6725 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:43.900378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354748d-fb76-4486-a529-a281dabac2a0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:43.919845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:43.920824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.921119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.921407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.922606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:43.924122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:43.926551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 30/55] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:43.929955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:43.931972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:43.935110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19594adf-de51-4e5b-8ea7-e4b76dfa6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:43.936140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19594adf-de51-4e5b-8ea7-e4b76dfa6dff None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:43.937129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19594adf-de51-4e5b-8ea7-e4b76dfa6dff None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:43.937129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19594adf-de51-4e5b-8ea7-e4b76dfa6dff None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:43.937462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19594adf-de51-4e5b-8ea7-e4b76dfa6dff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:43.938871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19594adf-de51-4e5b-8ea7-e4b76dfa6dff None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:44.006454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:44.029468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c6b20-22a3-4d16-ba36-0491fc04bcdf None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:44.043196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 26/56] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Jun 1 04:06:43 2022] POST /identity/v3/projects => generated 298 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:06:44.063716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:44.064416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:44.064711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:44.064998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:44.065864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.domain.id=default, user.id=c2efe34f830a400eb5eb8876be2a1da6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:44.066186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:44.067709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedac4c5-d7d9-47db-af53-527a030bcef4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:44.077684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 31/57] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Jun 1 04:06:43 2022] GET /identity/v3/role_assignments?user.id=c2efe34f830a400eb5eb8876be2a1da6&scope.domain.id=default&role.id=02d9e6449ea143beb203824112640698 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:46.314170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb201b-f83b-46d3-831b-c7788d2eb4c7 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:46.314728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb201b-f83b-46d3-831b-c7788d2eb4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:46.314728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb201b-f83b-46d3-831b-c7788d2eb4c7 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:46.316194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 27/58] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:46.322002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843578-974b-4447-a794-2e2597dea034 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:46.322563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843578-974b-4447-a794-2e2597dea034 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:46.322563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843578-974b-4447-a794-2e2597dea034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:46.340062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fb843578-974b-4447-a794-2e2597dea034 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:46.341337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fb843578-974b-4447-a794-2e2597dea034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:46.407883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 32/59] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:46.435725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb201b-f83b-46d3-831b-c7788d2eb4c7 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:46.541013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb201b-f83b-46d3-831b-c7788d2eb4c7 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:46.573629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4556b-09de-48d1-bec2-1d84424ebf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:46.574356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4556b-09de-48d1-bec2-1d84424ebf98 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:46.574645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4556b-09de-48d1-bec2-1d84424ebf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:46.574927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4556b-09de-48d1-bec2-1d84424ebf98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:46.576130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4556b-09de-48d1-bec2-1d84424ebf98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:46.577582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4556b-09de-48d1-bec2-1d84424ebf98 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:46.579172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 28/60] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:46.584693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843578-974b-4447-a794-2e2597dea034 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:46.650995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:46.681601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843578-974b-4447-a794-2e2597dea034 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:46.713119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b67cd-1ce6-4ad0-9894-d73aed698c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:46.713791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b67cd-1ce6-4ad0-9894-d73aed698c56 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:46.714082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b67cd-1ce6-4ad0-9894-d73aed698c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:46.714365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b67cd-1ce6-4ad0-9894-d73aed698c56 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:46.715092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b67cd-1ce6-4ad0-9894-d73aed698c56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:46.716587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b67cd-1ce6-4ad0-9894-d73aed698c56 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:46.830245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 33/61] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:06:46 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:50.267622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aaf66-bff3-48fd-aa96-100164817015 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.268621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aaf66-bff3-48fd-aa96-100164817015 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.269624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aaf66-bff3-48fd-aa96-100164817015 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.271177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 29/62] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:50.276685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.277549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.278027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.304720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:50.306336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:50.308060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2839ce-9e8b-4ccf-b698-e8786fe0343d None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.308380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2839ce-9e8b-4ccf-b698-e8786fe0343d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.308667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2839ce-9e8b-4ccf-b698-e8786fe0343d None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.310082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 30/63] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:06:50.316527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.317146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.317646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.348702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:50.351793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:50.418086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 34/64] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:50.436379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:50.460472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:50.461080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 31/65] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:06:50.485098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:50.553365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:50.603894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553dcf-53c3-45ac-9552-5ab1eba6b0f9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:50.617157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7a327-048a-43e1-9d9b-e7175b735ae4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:50.642280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ec16f-6f85-4bbf-b638-bfecbd570b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:50.643453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec16f-6f85-4bbf-b638-bfecbd570b6f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.644498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec16f-6f85-4bbf-b638-bfecbd570b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.644971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec16f-6f85-4bbf-b638-bfecbd570b6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.655736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dcf81-eb7e-42c1-b092-c6e48af9c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:50.655967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ec16f-6f85-4bbf-b638-bfecbd570b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:50.656888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dcf81-eb7e-42c1-b092-c6e48af9c632 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.657308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dcf81-eb7e-42c1-b092-c6e48af9c632 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.657738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dcf81-eb7e-42c1-b092-c6e48af9c632 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-127ec16f-6f85-4bbf-b638-bfecbd570b6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:50.665418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:50.666270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:50.667011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:50.669617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 35/66] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:50.669716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dcf81-eb7e-42c1-b092-c6e48af9c632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:50.671243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66837ef-12e7-4f0f-a8b2-c3c97d3423e3 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.671744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66837ef-12e7-4f0f-a8b2-c3c97d3423e3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.672228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66837ef-12e7-4f0f-a8b2-c3c97d3423e3 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.673762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 36/67] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:50.676118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66837ef-12e7-4f0f-a8b2-c3c97d3423e3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-a01dcf81-eb7e-42c1-b092-c6e48af9c632 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:50.682837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:50.683829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:06:50.684588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 32/68] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:06:50.690824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80b1d4-ab44-43eb-a5e3-cf5b56c4004e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.690824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80b1d4-ab44-43eb-a5e3-cf5b56c4004e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.690824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80b1d4-ab44-43eb-a5e3-cf5b56c4004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.703554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-4c80b1d4-ab44-43eb-a5e3-cf5b56c4004e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:50.704730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4c80b1d4-ab44-43eb-a5e3-cf5b56c4004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:50.743961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:50.778866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 33/69] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:50.787688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e21abc-54e6-4df4-a8e0-4ef7203e2294 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.787688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e21abc-54e6-4df4-a8e0-4ef7203e2294 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.787688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e21abc-54e6-4df4-a8e0-4ef7203e2294 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.787688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 34/70] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:50.787688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e21abc-54e6-4df4-a8e0-4ef7203e2294 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:50.806654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66837ef-12e7-4f0f-a8b2-c3c97d3423e3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:50.824962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:50.844180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:50.844605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.844894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.845172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.845925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:50.846238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:50.847722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74e546-292b-4797-9c56-481623daa8d6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:50.864873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 37/71] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:50.864873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.864873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.864873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.878337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:50.880064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:50.898885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e21abc-54e6-4df4-a8e0-4ef7203e2294 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:50.934315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:50.934973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.935265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.935596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.936322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:50.936633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:50.938025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dc186-e36c-49c4-8250-e65bd9670206 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:50.946799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 38/72] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:50.947269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 35/73] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:50.947660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:50.949053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e7ac-3e1a-4582-8004-2d1e7020fa7b None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:50.949472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e7ac-3e1a-4582-8004-2d1e7020fa7b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:50.949876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e7ac-3e1a-4582-8004-2d1e7020fa7b None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:50.951712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 36/74] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:50.953535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e7ac-3e1a-4582-8004-2d1e7020fa7b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:50.992262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.047801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68519589-ad21-47dd-9b3e-d7fa27faab4f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.058063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.058190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e7ac-3e1a-4582-8004-2d1e7020fa7b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.083935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.084750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.085035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.085313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.091449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca9f1e-eb7e-471c-bc4b-567499e1fc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.092320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9f1e-eb7e-471c-bc4b-567499e1fc53 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.092600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9f1e-eb7e-471c-bc4b-567499e1fc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.092879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9f1e-eb7e-471c-bc4b-567499e1fc53 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.095740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:51.104400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:51.105420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:51.106206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 39/75] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:06:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:51.106825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.111066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca9f1e-eb7e-471c-bc4b-567499e1fc53 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.112643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca9f1e-eb7e-471c-bc4b-567499e1fc53 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.114839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:50 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:51.116018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.116304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.116582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.132673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:51.133850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:51.198932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 38/77] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:51.200249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.208349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30cf3-c270-4e88-a781-b37e88cfaa1a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.225693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.243758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.244641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.244926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.245199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.246429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.247972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.249634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 40/78] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:51.250383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.284037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.305378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce61b50-970e-41b7-a3a3-9921e64616ef None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.338919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.339673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.339962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.340241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.349149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcec219-fdd1-44ad-93da-f6f99c79d476 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.350653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:51.358305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:51.359167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:06:51.359957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 39/79] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:51.360586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.387035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.387737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.388259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.388365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.389032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:51.389345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.390738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.398590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 41/80] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:51.399171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.442273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.460192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6130c20-309f-4b8b-b073-aba3c5f4cd5e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.493142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.493782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.494069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.494349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.497517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.501558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd974e8e-9fb4-4bc4-9bc2-563646ae0b08 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.536362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.537162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.537453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.537731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.538927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.540427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.542200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 42/81] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:51.542799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.562166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.563697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.644433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d13863-bcd5-48ac-99ae-2e48d1579b12 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.644575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.663087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 40/82] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:06:51.663952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.682029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.682695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.682983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.683267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.716211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.750579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.752113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.759881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaff0dd-3f71-4c27-b78c-a9d8a6480b9a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.793587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.794255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.794543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.794829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.795613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:51.795923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.797313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.805548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 41/83] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:51.806347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.850246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.854114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 43/84] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:06:51.854910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.904257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8feec-46f7-4496-a086-82f619d97883 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.919451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:51.937183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.938088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.938377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.938654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.939878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.941888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:51.943545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 42/85] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:51.944310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:51.954422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d125009-6ca9-419a-83bb-68b6e137a457 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:51.988055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:51.988726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:51.989016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:51.989293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:51.990003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:51.991490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.042061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770852d-6134-4db4-989f-74e39ebf223f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:52.054578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:52.061859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 44/86] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:06:51 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:06:52.062678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:52.077351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:52.078259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:52.078546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:52.078830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:52.080223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:52.082307 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.084160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 43/87] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:51 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:52.085071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:52.196226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51754fe-f993-4875-a8fb-6a00361dc747 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:52.200786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ff258-4cce-4621-a082-f78ba06e0bbf None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:52.228170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:52.238178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:52.239069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:52.241958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:52.241958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:52.241958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:52.242156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:52.242250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:52.242449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.242536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:52.242776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:52.244038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:52.244232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 44/88] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:52.245673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.249213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 45/89] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:52.249213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:52.249821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:52.352356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:52.424608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d63c78-3597-4924-8c7a-e8bb72887eab None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:52.431166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905ad4-4212-4d79-8d7e-2eccf2c7e4d5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:52.465476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:52.466052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:52.466344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:52.466628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:52.474340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:52.475283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:52.475795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:52.476292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:52.477745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=8ab699500cea4a518ae5ff2d36b49db6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:52.478236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:52.480689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.488857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 46/90] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/role_assignments?user.id=8ab699500cea4a518ae5ff2d36b49db6&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:52.489886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:52.588493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:52.590291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.612575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:52.692109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:52.712288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cec78-7e4f-4484-84a6-527142e5cc4c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:52.769793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 45/91] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:06:52.770573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:52.770695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:52.771371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:52.771667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:52.771952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:52.772802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=c2efe34f830a400eb5eb8876be2a1da6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:52.773123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:52.784994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:52.791361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/role_assignments?user.id=c2efe34f830a400eb5eb8876be2a1da6&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:52.792185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:53.021745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a746c-37ce-4b46-958b-65519ba3d792 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:53.025530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34256b-a822-4186-9a7a-c2f2f8a57991 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:53.065297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:53.096533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:53.097796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:53.098084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:53.098364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:53.118210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:53.119121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:53.119461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:53.119751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:53.124239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:53.125748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:53.130755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 48/93] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:53.133652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:53.147673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:53.255704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:53.257198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:53.343434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11921e-8082-4e64-9fb7-0ff9f829e1a6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:53.464554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 46/94] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:06:53.470881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:53.480094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:53.480782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:53.481077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:53.481362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:53.482226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] RBAC: Adding query filter params (role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=8ab699500cea4a518ae5ff2d36b49db6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:53.482546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:53.485822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:53.494162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 49/95] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:06:53 2022] GET /identity/v3/role_assignments?user.id=8ab699500cea4a518ae5ff2d36b49db6&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=a1af2eba782e475681e027a5d25335c6 => generated 259 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:53.544970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:53.601807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07e603-0251-4959-8f63-5b4173d9366a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:53.660745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bf049-0667-40e4-99bb-d262155ecf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:53.662141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bf049-0667-40e4-99bb-d262155ecf67 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:53.662647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bf049-0667-40e4-99bb-d262155ecf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:53.663174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bf049-0667-40e4-99bb-d262155ecf67 None admin] PATH_INFO: `/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:53.668311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bf049-0667-40e4-99bb-d262155ecf67 None admin] RBAC: Authorizing `identity:get_project(project_id=e1f7abc746404e9a9ab4e24cb7ff6884)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:53.670137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bf049-0667-40e4-99bb-d262155ecf67 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:53.672228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 47/96] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:53 2022] GET /identity/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:53.679163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:53.760133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:53.813169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dda6e-f6ef-46ab-ba7b-3f3f9f5451ac None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:53.848197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:53.848871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:53.849161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:53.849446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:53.850306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=e1f7abc746404e9a9ab4e24cb7ff6884, user.id=8ab699500cea4a518ae5ff2d36b49db6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:53.850627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:53.852209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a4f9-eb2b-46bd-93a4-9e965e069d47 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:53.862277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 50/97] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:06:53 2022] GET /identity/v3/role_assignments?user.id=8ab699500cea4a518ae5ff2d36b49db6&scope.project.id=e1f7abc746404e9a9ab4e24cb7ff6884&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:55.086770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb7b32-0269-4e8e-a145-fba10c3f58a3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.087281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb7b32-0269-4e8e-a145-fba10c3f58a3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.087379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb7b32-0269-4e8e-a145-fba10c3f58a3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.088817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 48/98] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:55.094330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687c363-da2f-4f14-9476-be48a7a0a2a0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.094741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687c363-da2f-4f14-9476-be48a7a0a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.094910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687c363-da2f-4f14-9476-be48a7a0a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.113252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d687c363-da2f-4f14-9476-be48a7a0a2a0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:55.114509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d687c363-da2f-4f14-9476-be48a7a0a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:55.192466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 51/99] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:06:55.230607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb7b32-0269-4e8e-a145-fba10c3f58a3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:55.341199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb7b32-0269-4e8e-a145-fba10c3f58a3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:55.359194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:55.378991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:55.379846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.380137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.380422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.392446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:55.396994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a01aa-f256-4e74-8387-c08573b73827 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.397293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a01aa-f256-4e74-8387-c08573b73827 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.397577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a01aa-f256-4e74-8387-c08573b73827 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.399108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 52/100] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:06:55.404645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.404940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.405222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:55.412625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:55.413558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:06:55.414334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 49/101] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:55.430172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:55.431383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:55.434901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:55.534178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:55.590491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 53/102] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:55.636237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:55.658353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b0f09-7586-473f-96b3-dd752f957474 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:55.731894 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:55.732623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.732912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.733195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.733966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:55.735644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:55.737074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:55.762744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 50/103] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:06:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:55.776244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:55.821397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:55.826126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262bd95-29bc-43e5-a357-21dcf1649321 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:55.915402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62e830-3d21-4090-a425-e459592c31e7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:55.915638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c5639-3496-43ec-a6bc-1deb5167e410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:55.915638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c5639-3496-43ec-a6bc-1deb5167e410 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.915638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c5639-3496-43ec-a6bc-1deb5167e410 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.915638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c5639-3496-43ec-a6bc-1deb5167e410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c5639-3496-43ec-a6bc-1deb5167e410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c5639-3496-43ec-a6bc-1deb5167e410 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 54/104] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73eaef-2242-42ff-8c63-4ba1abb203ab None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73eaef-2242-42ff-8c63-4ba1abb203ab None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73eaef-2242-42ff-8c63-4ba1abb203ab None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.915997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 55/105] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:55.916399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73eaef-2242-42ff-8c63-4ba1abb203ab None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:55.949055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ced5-2ff2-4f2e-a5ee-8790955c879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:55.950320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ced5-2ff2-4f2e-a5ee-8790955c879a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.951075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ced5-2ff2-4f2e-a5ee-8790955c879a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.952760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ced5-2ff2-4f2e-a5ee-8790955c879a None admin] PATH_INFO: `/v3/users/e3b2da81f95e42f0842705914d0fe90e` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.985046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ced5-2ff2-4f2e-a5ee-8790955c879a None admin] RBAC: Authorizing `identity:get_user(user_id=e3b2da81f95e42f0842705914d0fe90e)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:55.987161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ced5-2ff2-4f2e-a5ee-8790955c879a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:55.992767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 51/106] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:55 2022] GET /identity/v3/users/e3b2da81f95e42f0842705914d0fe90e => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:55.994469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:55.995257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:55.996921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:55.999005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:56.029437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:56.031049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:56.108719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73eaef-2242-42ff-8c63-4ba1abb203ab None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:56.155734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 52/107] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:56.157270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:56.186033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:56.188898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:56.189592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.189888 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.190167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.191531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:56.192994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:56.271058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25747a1-eae7-493f-b230-03b2111d5e60 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:56.303716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 56/108] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:06:55 2022] POST /identity/v3/projects => generated 302 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:06:56.304984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:56.316343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733859e3-d1fd-4fc1-ba13-ae5768d03f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:56.316343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733859e3-d1fd-4fc1-ba13-ae5768d03f6c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.316343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733859e3-d1fd-4fc1-ba13-ae5768d03f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.316343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733859e3-d1fd-4fc1-ba13-ae5768d03f6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.488957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733859e3-d1fd-4fc1-ba13-ae5768d03f6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:56.490394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733859e3-d1fd-4fc1-ba13-ae5768d03f6c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:56.513588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:56.519570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:56.523276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd42277-8ccf-42d8-bcc0-8eeeb6a0f40c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:56.561477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d6f60-c242-4ca9-b6cc-c3b2fcd57684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:56.562460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6f60-c242-4ca9-b6cc-c3b2fcd57684 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.562760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6f60-c242-4ca9-b6cc-c3b2fcd57684 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.563061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6f60-c242-4ca9-b6cc-c3b2fcd57684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.575639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6f60-c242-4ca9-b6cc-c3b2fcd57684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-c75d6f60-c242-4ca9-b6cc-c3b2fcd57684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:56.590027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:56.591032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:56.591908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 57/109] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:56.598174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de927b-4e12-4b34-b77e-aae1e9dbb535 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.598174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de927b-4e12-4b34-b77e-aae1e9dbb535 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.598174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de927b-4e12-4b34-b77e-aae1e9dbb535 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.598174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 58/110] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:56.598174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36884-13f3-4772-a9d4-7a8271a31b86 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.598174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36884-13f3-4772-a9d4-7a8271a31b86 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.609843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e36884-13f3-4772-a9d4-7a8271a31b86 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.609843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 59/111] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:56.609976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 53/112] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:06:56.610055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e36884-13f3-4772-a9d4-7a8271a31b86 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:56.610130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.610130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.610130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.628593 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:56.629789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:56.729296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 54/113] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:56.729492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e36884-13f3-4772-a9d4-7a8271a31b86 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:56.729975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:56.777951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:56.778634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.778920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.779202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.780097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:56.780406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:56.781974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ec0d2-78fe-4b68-8d00-ccbc6785d046 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:56.791729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 60/114] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:06:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:56.797630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.797928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.798215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.811683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:56.816080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:56.817413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:56.832474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:56.871720 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c805fb-fedc-4309-811c-46ed711451e0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:56.900755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 61/115] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:56.901947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:56.933312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:56.934193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:56.934482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:56.934765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:56.936125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:56.937596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:56.939512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 55/116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:56 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:56.941827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.052848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608844b1-1189-4979-99d4-ce49620d8dbb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.054946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2bbaf-c6f0-4c0c-9830-e8c2e1c9fb12 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.077859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:57.090292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.091386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.091794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.092198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.093591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:57.095205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:57.097105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 56/117] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:56 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:57.097970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.100342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.101709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.102959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.103428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.116432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:57.127620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:57.129255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:06:57.130207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:57.132986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 62/118] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:06:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:57.133898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.146701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:57.214858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab9ebb-b406-49a3-8d6a-830e09c736c5 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.248828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446abc09-6d2f-4cfc-a7fd-6e1be92fa9df None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.251662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.252470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.252759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.253561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.265147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:57.273676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:57.274528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:06:57.275328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:06:57.279678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 57/119] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:57.280518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.288538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.289205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.289632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.289925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.290780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] RBAC: Adding query filter params (role.id=3d876d008fdb464f93d245f9b7ef81c6, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=e3b2da81f95e42f0842705914d0fe90e) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:57.291103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:57.292719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:57.303759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 63/120] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/role_assignments?user.id=e3b2da81f95e42f0842705914d0fe90e&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=3d876d008fdb464f93d245f9b7ef81c6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:57.304406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.339834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:57.408969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecd18a-cff0-4162-ae14-b39d5077e157 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.409953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:57.449180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa39bf-17c8-4a78-82b5-d825cac2c9d6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.463830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d343b-e1a4-4696-9b85-0f47f88cc5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.465380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d343b-e1a4-4696-9b85-0f47f88cc5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.466349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d343b-e1a4-4696-9b85-0f47f88cc5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.466983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d343b-e1a4-4696-9b85-0f47f88cc5c9 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.468842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d343b-e1a4-4696-9b85-0f47f88cc5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:57.471105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d343b-e1a4-4696-9b85-0f47f88cc5c9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:57.474512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 58/121] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:57.476438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458e949-e2e6-432b-9c3a-a6a55209a1b7 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.477115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458e949-e2e6-432b-9c3a-a6a55209a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.477833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458e949-e2e6-432b-9c3a-a6a55209a1b7 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.484610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 59/122] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:57.484610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458e949-e2e6-432b-9c3a-a6a55209a1b7 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.501861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.503830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.504310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.505006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.506096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:57.506430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:57.508535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:57.517425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 64/123] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:57.522945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.613082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458e949-e2e6-432b-9c3a-a6a55209a1b7 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.636382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:57.652486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e323a4a-2921-41bd-854f-b0e708f35524 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.652773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.653397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.653690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.653974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.654738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:57.655053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:57.656783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482bc44-a9fc-41bc-9dc1-8edf59028dfb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:57.672760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 60/124] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:57.675661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.676434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.676820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.694173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:57.695527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:57.696480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:57.703686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:57.705040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:57.706167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:57.706664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/8ab699500cea4a518ae5ff2d36b49db6/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:57.709961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] RBAC: Authorizing `identity:create_grant(project_id=105f85dabeb0467da9127b26bd3c11cf, user_id=8ab699500cea4a518ae5ff2d36b49db6, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:57.712291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:57.768784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 61/125] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:57.769865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.783857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 65/126] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:06:57 2022] PUT /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/8ab699500cea4a518ae5ff2d36b49db6/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:06:57.790046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:57.973052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b7272-d758-40a3-ade7-1e4dd9ca8857 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:57.997359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae4bfd-ac2c-4c22-a620-a42107bfc440 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.025459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:58.030460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.031765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.032057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.032340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.035150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.035976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.039340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 66/127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:58.039340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.064122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.065344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.066404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.066881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.068363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.069929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.071787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 62/128] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:57 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:58.072695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.120610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:58.196188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca75868-6a00-4c2d-a166-6618b47450dd None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.234441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d66512-4d1e-4336-80b1-6fe21123960a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.260233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.261056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.261350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.261638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.280497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:06:58.288502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:06:58.289373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:06:58.290131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:06:58.290701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 67/129] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:06:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:06:58.291509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.303739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.304670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.304960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.317975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.317975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.317975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.317975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 63/130] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:58 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:58.317975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.397222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:58.492764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850a9da-c6dc-4efa-888f-c88f7d790f17 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.501884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2b4ba-8e14-45b1-b998-74ad1d52b77c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.525819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:58.542016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532df47-f671-477f-a840-879c1453caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.542897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532df47-f671-477f-a840-879c1453caee None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.543177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532df47-f671-477f-a840-879c1453caee None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.543510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532df47-f671-477f-a840-879c1453caee None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.544725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532df47-f671-477f-a840-879c1453caee None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.545794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.546315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532df47-f671-477f-a840-879c1453caee None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.546916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.547449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.547933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.548380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 68/131] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:58 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:58.549116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:06:58.549217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532df47-f671-477f-a840-879c1453caee None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.549884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.551997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.562191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 64/132] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:06:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:58.563271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.663670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532df47-f671-477f-a840-879c1453caee None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.678721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad0c91-3c45-4c2f-a3c8-06786409dbe2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:58.690040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:58.698765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184454-4408-4c39-874e-a185259f792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.699767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184454-4408-4c39-874e-a185259f792e None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.700112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184454-4408-4c39-874e-a185259f792e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.701339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0184454-4408-4c39-874e-a185259f792e None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/c2efe34f830a400eb5eb8876be2a1da6/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.702910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0184454-4408-4c39-874e-a185259f792e None admin] RBAC: Authorizing `identity:create_grant(project_id=105f85dabeb0467da9127b26bd3c11cf, user_id=c2efe34f830a400eb5eb8876be2a1da6, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.704529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0184454-4408-4c39-874e-a185259f792e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.716281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282246d-e7ae-4643-a29d-4159e34443f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:58.717309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282246d-e7ae-4643-a29d-4159e34443f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.717598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282246d-e7ae-4643-a29d-4159e34443f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.717886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282246d-e7ae-4643-a29d-4159e34443f4 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/8ab699500cea4a518ae5ff2d36b49db6/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.720554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282246d-e7ae-4643-a29d-4159e34443f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=105f85dabeb0467da9127b26bd3c11cf, user_id=8ab699500cea4a518ae5ff2d36b49db6, role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:58.723258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282246d-e7ae-4643-a29d-4159e34443f4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:58.805909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 69/133] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:06:58 2022] PUT /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/c2efe34f830a400eb5eb8876be2a1da6/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:06:58.808800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184454-4408-4c39-874e-a185259f792e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:58.828616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 65/134] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:06:58 2022] PUT /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/8ab699500cea4a518ae5ff2d36b49db6/roles/a1af2eba782e475681e027a5d25335c6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:06:58.830198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8738c-51a5-4662-b185-23fa592a8815 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.830496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8738c-51a5-4662-b185-23fa592a8815 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.830782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8738c-51a5-4662-b185-23fa592a8815 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.834377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 66/135] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:06:58.848930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:58.849240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:58.849508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:58.881095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:06:58.882619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:06:58.890641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:58.972967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0184454-4408-4c39-874e-a185259f792e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:59.030807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 67/136] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:06:59.036052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:59.036970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:59.037258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:59.037539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:59.038849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:59.040401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:59.042265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 70/137] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:06:58 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:59.047885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:59.069446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:59.194561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9554b-0629-4286-9323-1db156c9fd01 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:59.215020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:59.244593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:59.245417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:59.245709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:59.245992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:59.247222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:59.248749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:59.250388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 71/138] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:06:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:06:59.255900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:59.259290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:06:59.263716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7cbe4-cdd7-42ef-8b74-c3eb9dbd7256 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:59.312089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:59.313287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:59.313586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:59.313870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] PATH_INFO: `/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:59.316369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] RBAC: Authorizing `identity:get_project(project_id=e1f7abc746404e9a9ab4e24cb7ff6884)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:59.317910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:59.320940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 68/139] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:06:59 2022] GET /identity/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884 => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:06:59.327946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:06:59.402914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d3b3-9526-4c36-a081-3033cd1bd8e5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:59.445729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26f229-2a15-4d9f-8adc-fc11e05dc8a6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:06:59.477138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d07351-f293-495b-baae-1a5c21be6e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:59.478124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07351-f293-495b-baae-1a5c21be6e97 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:59.478603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07351-f293-495b-baae-1a5c21be6e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:59.479051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07351-f293-495b-baae-1a5c21be6e97 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:59.479984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d07351-f293-495b-baae-1a5c21be6e97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:59.481838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d07351-f293-495b-baae-1a5c21be6e97 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:59.497582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450bab3-bf90-4cb2-9497-17f1d6ba7a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:06:59.498888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450bab3-bf90-4cb2-9497-17f1d6ba7a2d None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:06:59.499406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450bab3-bf90-4cb2-9497-17f1d6ba7a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:06:59.499838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450bab3-bf90-4cb2-9497-17f1d6ba7a2d None admin] PATH_INFO: `/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884/users/8ab699500cea4a518ae5ff2d36b49db6/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:06:59.505204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450bab3-bf90-4cb2-9497-17f1d6ba7a2d None admin] RBAC: Authorizing `identity:create_grant(project_id=e1f7abc746404e9a9ab4e24cb7ff6884, user_id=8ab699500cea4a518ae5ff2d36b49db6, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:06:59.505204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450bab3-bf90-4cb2-9497-17f1d6ba7a2d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:06:59.603121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 72/140] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:06:59 2022] POST /identity/v3/users => generated 283 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:06:59.638035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 69/141] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:06:59 2022] PUT /identity/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884/users/8ab699500cea4a518ae5ff2d36b49db6/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:07:00.148601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9175738-6cdd-47a1-9216-fe966fa0e253 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.149233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9175738-6cdd-47a1-9216-fe966fa0e253 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.149738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9175738-6cdd-47a1-9216-fe966fa0e253 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.151334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 73/142] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:00.158133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.158680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.159730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.178857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:00.180409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:00.209400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:00.259446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 70/143] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:00.279377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9175738-6cdd-47a1-9216-fe966fa0e253 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:00.331157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:00.393040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9175738-6cdd-47a1-9216-fe966fa0e253 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:00.431168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:00.431968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.432253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.432527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.443188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:00.451153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:00.452057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:00.452825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 74/144] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:00.458671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:00.560983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fff5df-6d3c-4224-9530-cc8cf09e99a5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:00.597574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:00.598253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.598534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.598810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.599596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:00.599904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:00.601312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1cc092-1e6e-4fab-b3bc-1c72af1f06d5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:00.610572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 71/145] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:00.616253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:00.711369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af63ac0-ebc2-486d-bc5c-c0c9fcb1835f None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.711549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af63ac0-ebc2-486d-bc5c-c0c9fcb1835f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.711814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af63ac0-ebc2-486d-bc5c-c0c9fcb1835f None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.713179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 72/146] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:00.718503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.718791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.719065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.724389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:00.729790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0c67b-fe31-4bf3-9342-53893b52fcb7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:00.753640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:00.755213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:00.786304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:00.808889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:00.810139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:00.810722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:00.811198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] PATH_INFO: `/v3/users/e3b2da81f95e42f0842705914d0fe90e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:00.813285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] RBAC: Authorizing `identity:get_user(user_id=e3b2da81f95e42f0842705914d0fe90e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:00.814983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:00.817580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 75/147] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:00 2022] GET /identity/v3/users/e3b2da81f95e42f0842705914d0fe90e => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:00.824589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:00.874361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 73/148] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:00.902424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:01.024706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d14f23-f752-42d2-8759-4c08ac5faca5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:01.082888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc650bba-a6dd-475e-a557-1916321ad7cc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:01.097372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:01.098577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.099064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.099581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.102682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:01.104422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:01.106366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 76/149] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:00 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:01.113559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:01.154308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:01.156008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.156519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.157010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.181262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:01.192132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:01.193054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:01.193810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:01.200097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:01.202123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 74/150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:01.212593 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:01.236944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:01.326361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbeede-f08c-4c43-84cb-64cb0c2511a8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:01.367711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771704c-160d-4d43-bae3-2d8469e8e5d2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:01.405047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:01.405716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.406004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.406343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/e3b2da81f95e42f0842705914d0fe90e/roles/3d876d008fdb464f93d245f9b7ef81c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.408395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=105f85dabeb0467da9127b26bd3c11cf, user_id=e3b2da81f95e42f0842705914d0fe90e, role_id=3d876d008fdb464f93d245f9b7ef81c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:01.409896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:01.420311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:01.421328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.421811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.422326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.423396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:01.423907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:01.435452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:01.441277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 75/151] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:01.447432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:01.510420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 77/152] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:07:01 2022] PUT /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf/users/e3b2da81f95e42f0842705914d0fe90e/roles/3d876d008fdb464f93d245f9b7ef81c6 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:01.543554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:01.579223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc4842-17d7-4c45-86fc-e46904ebae15 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:01.636726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d86b7-b599-4883-9a0e-1229f12a3196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:01.637643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d86b7-b599-4883-9a0e-1229f12a3196 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.637929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d86b7-b599-4883-9a0e-1229f12a3196 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.638205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d86b7-b599-4883-9a0e-1229f12a3196 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.640563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d86b7-b599-4883-9a0e-1229f12a3196 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:01.642068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d86b7-b599-4883-9a0e-1229f12a3196 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:01.643915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 76/153] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:01 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:01.649495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:01.805966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:01.812810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba791309-c619-4420-b7e4-bb1e312bd1f7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:01.853691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:01.854291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.854716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.855000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.948841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:01.950353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:01.951942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1c4e5-0fa9-44ae-bf23-4b7a9f3a26f2 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.952234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1c4e5-0fa9-44ae-bf23-4b7a9f3a26f2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.952514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1c4e5-0fa9-44ae-bf23-4b7a9f3a26f2 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:01.953895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 77/154] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:01.959736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:01.960028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:01.960338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:02.004837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:02.006050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:02.084332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:02.094660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 78/155] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:02.107653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:02.153778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 78/156] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:02.194347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:02.282542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b1fa0-67e5-4ff6-9b8a-7520f70831da None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:02.363329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:02.364856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:02.366466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:02.366903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:02.367350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:02.368730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:02.370342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:02.372458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 79/157] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:02 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:02.383885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:02.418371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e65e96-f5ad-428f-a598-b9b56c37e8da None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:02.475874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:02.477173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:02.477456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:02.477729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:02.496218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:02.505408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:02.506280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:02.507340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 79/158] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:02.514924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:02.541643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:02.664494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96471cc4-d1c5-476a-bcfc-1843a90764b3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:02.665224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afd3fd-62ba-4558-9070-3b079e433f0d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:02.697002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:02.704023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:02.705153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:02.705592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:02.711396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:02.711396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:02.711396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:02.711396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:02.715597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:02.716272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:02.716556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:02.716831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:02.717676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=8ab699500cea4a518ae5ff2d36b49db6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:02.718033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:02.720312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 80/159] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:02.723549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:02.729688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:02.740622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 80/160] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:02 2022] GET /identity/v3/role_assignments?user.id=8ab699500cea4a518ae5ff2d36b49db6&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 608 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:02.841378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cc965-fa13-4701-96f9-544b7d6a358b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:02.879867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dbd96-53bf-41c5-a909-cde097008894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:02.881136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dbd96-53bf-41c5-a909-cde097008894 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:02.881608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dbd96-53bf-41c5-a909-cde097008894 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:02.882354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dbd96-53bf-41c5-a909-cde097008894 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:02.883788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dbd96-53bf-41c5-a909-cde097008894 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:02.885475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dbd96-53bf-41c5-a909-cde097008894 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:02.887464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 81/161] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:02 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:02.896451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:03.014569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda94818-3ec5-40de-9614-c0a8ca304e6d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:03.044284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:03.053761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:03.066044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.066044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.066044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.133959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:03.135922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:03.246124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc52cb-68c4-4bd2-befc-d76573ee8539 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.246735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc52cb-68c4-4bd2-befc-d76573ee8539 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.247204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc52cb-68c4-4bd2-befc-d76573ee8539 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.248790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 82/162] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:03.254704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.260252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.260252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.282464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 81/163] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:03.287154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:03.288433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:03.291719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:03.339709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:03.406766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:03.433636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 83/164] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:03.467412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:03.473695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e2779-0165-4121-8666-cd8b71055573 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:03.532018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:03.534699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.535182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.535669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.538791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:03.540527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:03.542455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 82/165] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:03 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:03.548279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:03.687471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256982d-b080-4693-97b9-eefc46332f8e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:03.730834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:03.731684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.731972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.732253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.740296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:03.751415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20296858-de6d-474b-8c58-50062a4c9e26 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:03.752949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:03.765092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:03.769773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:03.770747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 84/166] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:03.773032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:03.814821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:03.816069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.816355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.816631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.817489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] RBAC: Adding query filter params (role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=8ab699500cea4a518ae5ff2d36b49db6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:03.817806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:03.819283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:03.833680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 83/167] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:03 2022] GET /identity/v3/role_assignments?user.id=8ab699500cea4a518ae5ff2d36b49db6&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=a1af2eba782e475681e027a5d25335c6 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:03.856940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:03.933674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ece38f-d75b-46db-bc40-4fdc0b01fbe3 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:03.974939 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:03.975573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:03.975862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:03.976144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:03.976928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:03.977239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:03.978696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:03.988447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 85/168] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:03.994085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:04.267780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eea01-cdf9-4e92-a8e4-ddc16dbcabf2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:04.307489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:04.308379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.308664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.309047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] PATH_INFO: `/v3/users/c2efe34f830a400eb5eb8876be2a1da6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.310270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c2efe34f830a400eb5eb8876be2a1da6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:04.311765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:04.319684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 84/169] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:03 2022] GET /identity/v3/users/c2efe34f830a400eb5eb8876be2a1da6 => generated 247 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:04.325284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:04.448691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d4670-1d79-4784-a0ff-0a282e166e9a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:04.462450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:04.487974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fed2b-f64f-4704-9238-def077577f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:04.488639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fed2b-f64f-4704-9238-def077577f0b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.488924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fed2b-f64f-4704-9238-def077577f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.489204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fed2b-f64f-4704-9238-def077577f0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.578933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fed2b-f64f-4704-9238-def077577f0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:04.580471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fed2b-f64f-4704-9238-def077577f0b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:04.708135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 86/170] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:04.713910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:04.716183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6d15f-6360-4b25-813e-2c2ff861dde1 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.716478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6d15f-6360-4b25-813e-2c2ff861dde1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.716749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6d15f-6360-4b25-813e-2c2ff861dde1 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.718112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 87/171] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:04.726142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7070-bf26-4cf0-88c4-d1b5d04668ff None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.726431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7070-bf26-4cf0-88c4-d1b5d04668ff None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.726718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7070-bf26-4cf0-88c4-d1b5d04668ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.754493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5c2e7070-bf26-4cf0-88c4-d1b5d04668ff None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:04.755728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5c2e7070-bf26-4cf0-88c4-d1b5d04668ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:04.769348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:04.789050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:04.864205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7867b-35c1-42e2-8e7e-027e2e4f98b6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:04.873840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 88/172] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:04.875390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa139-a967-4be1-9a48-1a50118f7a9a None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.875687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa139-a967-4be1-9a48-1a50118f7a9a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.875968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa139-a967-4be1-9a48-1a50118f7a9a None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.877288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 89/173] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:04.884729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.885018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.885299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.901251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f197f46-0666-4144-a544-97373deee53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:04.902117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f197f46-0666-4144-a544-97373deee53e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:04.902401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f197f46-0666-4144-a544-97373deee53e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:04.902680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f197f46-0666-4144-a544-97373deee53e None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:04.903916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f197f46-0666-4144-a544-97373deee53e None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:04.905395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f197f46-0666-4144-a544-97373deee53e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:04.907094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 85/174] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:04 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:04.912905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f197f46-0666-4144-a544-97373deee53e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:04.917924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:04.919109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:05.037562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 90/175] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:05.038430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.115721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.125387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f197f46-0666-4144-a544-97373deee53e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.137221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.167429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf4ae1-4b1c-4dde-a802-1856c4c3ae49 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.196044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.196723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.197014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.197302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.198154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=c2efe34f830a400eb5eb8876be2a1da6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:05.198474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:05.199960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:05.204529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.205333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.205615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.205924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.206295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 86/176] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:04 2022] GET /identity/v3/role_assignments?user.id=c2efe34f830a400eb5eb8876be2a1da6&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:05.207110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.218354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:05.226534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:05.227462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:05.228258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 91/177] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:05.233999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.376374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902a9f5-d4dd-43b8-b96c-7028b4d0b01b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.388897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096e2c7-fe60-4a8e-8ab2-79ce3f2c352c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.442264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.446567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fd9f6-86f0-4246-80b4-47896cc19394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.448360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd9f6-86f0-4246-80b4-47896cc19394 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.454159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.454627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd9f6-86f0-4246-80b4-47896cc19394 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.454627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fd9f6-86f0-4246-80b4-47896cc19394 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.454757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.454757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.454757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.454757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:05.454757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:05.454757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a5dcc-b4d6-4553-aaed-c7ec8104e59c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:05.463151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fd9f6-86f0-4246-80b4-47896cc19394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:05.464241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 92/178] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:05.465411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8944-79c2-4587-9d27-57fcfe18a56e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.465714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8944-79c2-4587-9d27-57fcfe18a56e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.466002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8944-79c2-4587-9d27-57fcfe18a56e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.467242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 93/179] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:05.468832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a17f4-c349-4720-a0c4-08926b466c41 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.469120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a17f4-c349-4720-a0c4-08926b466c41 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.469401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a17f4-c349-4720-a0c4-08926b466c41 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.470865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 94/180] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:05.473062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a17f4-c349-4720-a0c4-08926b466c41 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-183fd9f6-86f0-4246-80b4-47896cc19394 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:05.476435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:05.477309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:05.478505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 87/181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:05.480237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cbb61-d72c-4dc8-9daf-f50dce778036 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.480527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cbb61-d72c-4dc8-9daf-f50dce778036 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.480809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cbb61-d72c-4dc8-9daf-f50dce778036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.500732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c04cbb61-d72c-4dc8-9daf-f50dce778036 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:05.502807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c04cbb61-d72c-4dc8-9daf-f50dce778036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:05.520792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.583282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 88/182] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:05.584976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.585269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.585550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.602213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:05.603475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:05.616383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a17f4-c349-4720-a0c4-08926b466c41 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.653509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.654415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.654706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.654992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] PATH_INFO: `/v3/users/50ec687901b94804bfa7acfab0abeadb` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.672426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] RBAC: Authorizing `identity:get_user(user_id=50ec687901b94804bfa7acfab0abeadb)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:05.674032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:05.676471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 95/183] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/users/50ec687901b94804bfa7acfab0abeadb => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:05.681142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.687268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 89/184] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:05.688411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.741955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.782828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.792736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568b3ca-0680-4c7a-b80d-74888b5c8f7a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.803222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aea67-d6fa-4e16-a965-1e6ed0f6cac1 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.830285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.831595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.831925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.832210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.832961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:05.833269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:05.834667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:05.838161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.838967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.839249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.839554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.840736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:05.842178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:05.843485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 96/185] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:05.843891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 90/186] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:05.844715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.848981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:05.952874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33decaf-bb01-4d2d-8395-7370000e26fb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.953588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c97fc-8d7f-4223-a762-5a2c1c456b56 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:05.967966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:05.989742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb38aa0-2543-41d1-924d-6bc17db3034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.990387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb38aa0-2543-41d1-924d-6bc17db3034a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.990495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:05.990670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb38aa0-2543-41d1-924d-6bc17db3034a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.990944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb38aa0-2543-41d1-924d-6bc17db3034a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:05.991283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:05.991665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:05.991954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.002224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:06.010736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:06.011650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:06.012509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 91/187] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:06.013165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.027632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.060345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb38aa0-2543-41d1-924d-6bc17db3034a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.061805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb38aa0-2543-41d1-924d-6bc17db3034a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.114401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d893b-102b-4eaf-af4c-e8aebac30b59 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.148839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.149710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.149998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.150280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] PATH_INFO: `/v3/users/8ab699500cea4a518ae5ff2d36b49db6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.151513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab699500cea4a518ae5ff2d36b49db6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.152963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.154564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 92/188] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/users/8ab699500cea4a518ae5ff2d36b49db6 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:06.155321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.165454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 97/189] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:06.165454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fdf19-e356-45b2-b9cf-5364a4a33555 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.165454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fdf19-e356-45b2-b9cf-5364a4a33555 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.165454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fdf19-e356-45b2-b9cf-5364a4a33555 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.165454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 98/190] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:06.165454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fdf19-e356-45b2-b9cf-5364a4a33555 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.181924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.234416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.256767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b3927-0cc9-4793-8d17-a6bcb55df804 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.269871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fdf19-e356-45b2-b9cf-5364a4a33555 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.290331 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.291000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.291283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.291591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.292305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.293751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.302873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.303573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.305161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.305161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.305161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:06.305161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.306512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.314483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 99/191] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:06.315795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.381528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 93/192] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:06 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:06.381528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.392850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.421822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d83aa0-5b47-4dc3-a9e3-b051916da0bd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.461724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.462506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.462693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.462980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.506448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348aa31d-5883-4f86-96aa-57f2fed85b49 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.537293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.542648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.543565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207545-7834-47d0-89c7-ad1fed9f3859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.544317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.544481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207545-7834-47d0-89c7-ad1fed9f3859 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.544728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207545-7834-47d0-89c7-ad1fed9f3859 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.545137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207545-7834-47d0-89c7-ad1fed9f3859 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.546300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207545-7834-47d0-89c7-ad1fed9f3859 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.547800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207545-7834-47d0-89c7-ad1fed9f3859 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.550113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 94/193] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:06.556396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.556678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.556960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.576189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:06.577727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:06.644630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.664606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 95/194] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:06.665471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.676614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 100/195] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:06.678565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.798499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb2ec-1302-4a12-8409-692e4fca8a16 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.799012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b34cb-f631-4e7b-87e9-d22e182e7f6b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.837613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.839465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.839934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.840218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.840503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.841339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=50ec687901b94804bfa7acfab0abeadb) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:06.841659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.843127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.851756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:06.852645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:06.852933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:06.853213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] PATH_INFO: `/v3/users/50ec687901b94804bfa7acfab0abeadb` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:06.854074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 101/196] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/role_assignments?user.id=50ec687901b94804bfa7acfab0abeadb&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:06.854449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] RBAC: Authorizing `identity:get_user(user_id=50ec687901b94804bfa7acfab0abeadb)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:06.855490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.855944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:06.857753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 96/197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/users/50ec687901b94804bfa7acfab0abeadb => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:06.858402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:06.906181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:06.969563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeb316-b9a8-4acf-92f4-ace470522a99 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:06.980243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e5ac3-61f3-43db-bec0-fad5a272bab7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.008354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.009289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.009516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.009800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] PATH_INFO: `/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.011031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] RBAC: Authorizing `identity:get_project(project_id=e1f7abc746404e9a9ab4e24cb7ff6884)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.012539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:07.014282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 102/198] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/projects/e1f7abc746404e9a9ab4e24cb7ff6884 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:07.015022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.022055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.023230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.023786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.024353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.037019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:07.045991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:07.046929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:07.047762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:07.051208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 97/199] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:07.052320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.129595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0bbcc-7104-4f92-815d-627522e68c36 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.132154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:07.153820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:07.183463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea41e19-1eea-4da1-8925-f55ee2c47dc1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.220900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.222037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.222591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.223111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.224256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=e1f7abc746404e9a9ab4e24cb7ff6884, user.id=8ab699500cea4a518ae5ff2d36b49db6) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:07.224813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.226551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:07.227983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.228711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.229197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.229350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.235239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 98/200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/role_assignments?user.id=8ab699500cea4a518ae5ff2d36b49db6&scope.project.id=e1f7abc746404e9a9ab4e24cb7ff6884&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:07.236383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.467936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe19d27-43d2-4ef3-b163-5205c0c76916 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.487835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.489375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:07.503445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:07.527334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.528497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.529008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.529486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.530458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:07.531066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.532700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:07.544999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 99/201] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:07.556706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.692517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 103/202] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:07.698291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.712464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cc1c8-d417-4509-9e38-c038e2eadda7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.717654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:07.752977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.753787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.754073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.754350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] PATH_INFO: `/v3/users/e3b2da81f95e42f0842705914d0fe90e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.756029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] RBAC: Authorizing `identity:get_user(user_id=e3b2da81f95e42f0842705914d0fe90e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.757763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:07.759403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 100/203] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/users/e3b2da81f95e42f0842705914d0fe90e => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:07.765483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.820931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:07.849774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a32bf-d113-48f9-abcf-48e4395413d3 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.870023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a21bb-1e5c-493f-a7b5-33d0a4157a37 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:07.906704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.907406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.907698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.908079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.908649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:07.910222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:07.910508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:07.910789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:07.916070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.917588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:07.919832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 104/204] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:07.925967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:07.980954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:07.982442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:08.055916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:08.087192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 101/205] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:08.093079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:08.104643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:08.135792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f259d1-0cfa-4c8a-a98a-8f2e171da976 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:08.211128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c590ba2-6f4c-43eb-b945-f493d6b91a96 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:08.211984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:08.212765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:08.213060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:08.213458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:08.214301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] RBAC: Adding query filter params (role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=50ec687901b94804bfa7acfab0abeadb) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:08.214620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:08.216122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:08.226301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 105/206] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:07 2022] GET /identity/v3/role_assignments?user.id=50ec687901b94804bfa7acfab0abeadb&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=a1af2eba782e475681e027a5d25335c6 => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:08.263649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9b3a7-5f27-49b7-b1c1-7888901328b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:08.264661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9b3a7-5f27-49b7-b1c1-7888901328b7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:08.264959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9b3a7-5f27-49b7-b1c1-7888901328b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:08.265242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9b3a7-5f27-49b7-b1c1-7888901328b7 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:08.266789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9b3a7-5f27-49b7-b1c1-7888901328b7 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:08.268981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9b3a7-5f27-49b7-b1c1-7888901328b7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:08.271084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 102/207] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:08 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:08.277239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:08.421869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2752-4528-414e-9b9e-913c869f1ef0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:08.458685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:08.459732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:08.460214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:08.460702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:08.461761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] RBAC: Adding query filter params (role.id=3d876d008fdb464f93d245f9b7ef81c6, scope.project.id=105f85dabeb0467da9127b26bd3c11cf, user.id=e3b2da81f95e42f0842705914d0fe90e) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:08.462271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:08.464021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adf9b3-38aa-40dd-9841-759733961170 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:08.471129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 106/208] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:08 2022] GET /identity/v3/role_assignments?user.id=e3b2da81f95e42f0842705914d0fe90e&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=3d876d008fdb464f93d245f9b7ef81c6 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:09.051267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d0011-df0b-4f14-b5a2-abd2bdaaea2e None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.051800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d0011-df0b-4f14-b5a2-abd2bdaaea2e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.051874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d0011-df0b-4f14-b5a2-abd2bdaaea2e None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.053386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 103/209] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:09.058949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.059402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.059574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.065656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:09.077128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:09.078443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:09.152627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 107/210] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:09.178270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d0011-df0b-4f14-b5a2-abd2bdaaea2e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:09.200253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:09.294947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d0011-df0b-4f14-b5a2-abd2bdaaea2e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:09.333929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b8284-cbc5-45fb-a55a-70fdab43bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:09.334663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b8284-cbc5-45fb-a55a-70fdab43bb61 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.334946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b8284-cbc5-45fb-a55a-70fdab43bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.335221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b8284-cbc5-45fb-a55a-70fdab43bb61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.336439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b8284-cbc5-45fb-a55a-70fdab43bb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:09.338013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b8284-cbc5-45fb-a55a-70fdab43bb61 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:09.340069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 104/211] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:09.345331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:09.350028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8109bf-481b-4f43-8925-ebf71c1e41d3 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.350028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8109bf-481b-4f43-8925-ebf71c1e41d3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.350028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8109bf-481b-4f43-8925-ebf71c1e41d3 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.350028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 105/212] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:09.357025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.357544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.357825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.378639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:09.380309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:09.471898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 106/213] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:09.483112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6921a6-94c4-48eb-a65a-effae16c6a8e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:09.497468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:09.520861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:09.524827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:09.526062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.526820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.527675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.528438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:09.529911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:09.551719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:09.617161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce161b8b-3e25-40d7-b8c9-c241ccaa5dfa None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:09.640763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 108/214] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:09 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:09.672217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4c885-2fad-47d5-b0a9-dd4917e653c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:09.673475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c885-2fad-47d5-b0a9-dd4917e653c6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.673963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c885-2fad-47d5-b0a9-dd4917e653c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.674515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c885-2fad-47d5-b0a9-dd4917e653c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.688046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4c885-2fad-47d5-b0a9-dd4917e653c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2dd4c885-2fad-47d5-b0a9-dd4917e653c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:09.696779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:09.697711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:09.698483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:09.701260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 107/215] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:09.708415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:09.743207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee96a5-305a-405c-b5de-df3e419c1c8c None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.743846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee96a5-305a-405c-b5de-df3e419c1c8c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.744324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee96a5-305a-405c-b5de-df3e419c1c8c None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.746263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 108/216] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:09.752790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.753442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.754019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.773479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:09.775049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:09.852561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de5b03-fa37-4dbd-841e-48d135a35e77 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:09.862589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 109/217] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:09.884774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:09.896326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:09.896863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:09.898044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:09.898478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:09.899026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:09.900056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:09.900505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:09.902196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:09.911759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 109/218] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:09.917336 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:09.989914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.011926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b76593-4441-4277-82d3-1d82d81123e1 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.051639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f1983-cabd-4aed-943b-7ecec3d8abe6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.054100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.055015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.055335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.055613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.070126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:10.078167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:10.079058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:10.079906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 110/219] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:10.086819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.096803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.097845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.098071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.098351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] PATH_INFO: `/v3/users/4445867b60454175a1b82e451221cb86` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.114266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] RBAC: Authorizing `identity:get_user(user_id=4445867b60454175a1b82e451221cb86)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:10.115806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:10.117787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 110/220] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:09 2022] GET /identity/v3/users/4445867b60454175a1b82e451221cb86 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:10.129650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.166242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.211446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e86b-b7c8-473c-92f0-57ddfd534e45 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.245000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a334ad0-611d-4e25-9b53-63206a6e4e99 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.258631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.260692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.261436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.261732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.262001 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.262847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:10.263162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:10.264605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:10.279455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 111/221] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:10.285124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.287424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.288086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.288368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.288641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.388316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:10.389826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:10.494474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cdcd2-4cea-4c93-88ea-ae516a95aeac None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.515672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 111/222] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:10.524774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.559655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.563040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.573813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.576281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.576281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.576281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] PATH_INFO: `/v3/users/50ec687901b94804bfa7acfab0abeadb` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.577847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] RBAC: Authorizing `identity:get_user(user_id=50ec687901b94804bfa7acfab0abeadb)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:10.579322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:10.588880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 112/223] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:10 2022] GET /identity/v3/users/50ec687901b94804bfa7acfab0abeadb => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:10.589716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.748153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832f45c-db46-4fef-9ac3-4211e878eaa2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.750770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a609de-f712-4820-abdc-2ba402f434d5 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.788942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.789823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.790102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.790371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.795483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:10.798122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:10.803382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 113/224] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:10 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:10.816480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.825494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.826841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.827127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.827434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.830394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:10.832307 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:10.834941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 112/225] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:10 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:10.842228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:10.915623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.952510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7cf8d-57a4-4ce9-93fd-a3816e21a1d5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.967587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:10.994084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:10.994979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:10.995215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5357df-0ae3-4cff-bd5d-7dd2401768f8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:10.995342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:10.995573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/50ec687901b94804bfa7acfab0abeadb/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:10.998608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a32ec861d2104c71ad90b203e9ae984c, user_id=50ec687901b94804bfa7acfab0abeadb, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:11.000083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:11.031835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:11.032543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.032832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.033111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.034552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=4445867b60454175a1b82e451221cb86) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:11.034865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:11.036337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c4106-138b-4009-9371-19e5138fc3c8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:11.042312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 113/226] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:10 2022] GET /identity/v3/role_assignments?user.id=4445867b60454175a1b82e451221cb86&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:11.048304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32093a29-3eba-49ad-9efe-4303b933b9ff None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.048897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32093a29-3eba-49ad-9efe-4303b933b9ff None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.049420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32093a29-3eba-49ad-9efe-4303b933b9ff None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.055616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 114/227] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:11 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:11.057562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.058100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.058574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.074154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 114/228] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:07:10 2022] PUT /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/50ec687901b94804bfa7acfab0abeadb/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:11.097156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:11.098409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:11.217092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 115/229] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:11.256184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:11.388756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5943b64-c8e8-4452-86d2-a4e2f110e6b6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:11.416678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:11.426272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:11.427425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.427949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.428428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.440914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:11.449770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:11.450647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:11.451488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:11.453810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 115/230] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:11.461909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:11.538029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:11.570116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd9ba5-6fd3-412e-ace6-de5d90a6112f None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:11.607180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:11.608252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.608747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.609210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.610147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:11.610654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:11.612300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:11.621028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 116/231] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:11.626864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:11.731824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e00b67-4b5c-46b8-9c1b-8376324a19df None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:11.768557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:11.769451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.769737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.770012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] PATH_INFO: `/v3/users/50ec687901b94804bfa7acfab0abeadb` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.771207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] RBAC: Authorizing `identity:get_user(user_id=50ec687901b94804bfa7acfab0abeadb)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:11.772700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:11.774307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 116/232] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:11 2022] GET /identity/v3/users/50ec687901b94804bfa7acfab0abeadb => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:11.779884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:11.887757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873b28-9049-4a6f-86d6-15402cfecc4f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:11.922624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612538f1-0a97-415d-855d-667dde70da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:11.923520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612538f1-0a97-415d-855d-667dde70da22 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:11.923802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612538f1-0a97-415d-855d-667dde70da22 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:11.924076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612538f1-0a97-415d-855d-667dde70da22 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:11.925833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612538f1-0a97-415d-855d-667dde70da22 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:11.927291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612538f1-0a97-415d-855d-667dde70da22 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:11.928992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 117/233] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:11 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:11.934650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:11.994047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:12.042560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4df816-7a8d-4dc1-b01f-0776a5cf9803 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:12.077920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b7581-9599-4a4d-a16b-2200ff4d28f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:12.078815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b7581-9599-4a4d-a16b-2200ff4d28f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:12.079092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b7581-9599-4a4d-a16b-2200ff4d28f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:12.079396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b7581-9599-4a4d-a16b-2200ff4d28f0 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/50ec687901b94804bfa7acfab0abeadb/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:12.081744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b7581-9599-4a4d-a16b-2200ff4d28f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a32ec861d2104c71ad90b203e9ae984c, user_id=50ec687901b94804bfa7acfab0abeadb, role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:12.083214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b7581-9599-4a4d-a16b-2200ff4d28f0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:12.201943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 117/234] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:07:11 2022] PUT /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/50ec687901b94804bfa7acfab0abeadb/roles/a1af2eba782e475681e027a5d25335c6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:12.625009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd46298-a282-4e90-ae4e-9fbfe0602a5f None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:12.625460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd46298-a282-4e90-ae4e-9fbfe0602a5f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:12.625638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd46298-a282-4e90-ae4e-9fbfe0602a5f None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:12.627190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 118/235] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:12.633195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:12.633485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:12.633768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:12.641972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2e1b9-84c0-46c1-a4fe-0ce3427789a8 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:12.642270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2e1b9-84c0-46c1-a4fe-0ce3427789a8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:12.642830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2e1b9-84c0-46c1-a4fe-0ce3427789a8 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:12.645243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 119/236] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:12.655323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:12.655598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:12.655872 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:12.659539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:12.660759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:12.662595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:12.682434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:12.683652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:12.751803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 118/237] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:12.773555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 120/238] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:12.808507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:12.810686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:12.867390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:12.911768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7f36c-d8d3-48b3-844e-5de95a7901d1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:12.931707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d369de-cc35-46d7-9a78-1ede6e768208 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:12.949829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:12.950857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:12.951548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:12.952101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:12.959508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:12.972456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:12.983200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:12.984099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:12.984864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:12.987679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 121/239] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:12.993292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:13.020021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:13.020828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.021282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.021394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.022552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:13.024203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:13.025981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 119/240] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:13.031507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:13.170110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e6fc3-50ad-47f5-9287-b8251193e6fe None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:13.207425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57964ff7-7a58-4520-9d18-ade74040e8ba None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:13.226263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:13.240900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:13.241585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.255408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.255408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.255408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:13.255408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:13.255408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:13.255777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:13.255777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.255777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.255777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.256027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:13.256090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:13.264562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 122/241] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:13.270211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:13.335611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:13.382725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72ef3-06c6-4020-b5e1-50ce819558af None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:13.396299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 120/242] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:13 2022] POST /identity/v3/users => generated 293 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:07:13.434524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:13.435370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.439884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.440185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] PATH_INFO: `/v3/users/5c8ee13c6c4f4fe39bc3648bf2d4543d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.460666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] RBAC: Authorizing `identity:get_user(user_id=5c8ee13c6c4f4fe39bc3648bf2d4543d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:13.462267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:13.464518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 123/243] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:13 2022] GET /identity/v3/users/5c8ee13c6c4f4fe39bc3648bf2d4543d => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:13.469823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:13.634640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d4f58-4ca3-4f95-afc4-3bfaa1e7adaa None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:13.677953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f572cc6-3cf4-4dd9-86d4-4a55de443a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:13.678615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f572cc6-3cf4-4dd9-86d4-4a55de443a07 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.678904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f572cc6-3cf4-4dd9-86d4-4a55de443a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.679183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f572cc6-3cf4-4dd9-86d4-4a55de443a07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.719345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:13.756494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f572cc6-3cf4-4dd9-86d4-4a55de443a07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:13.758243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f572cc6-3cf4-4dd9-86d4-4a55de443a07 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:13.872996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 121/244] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:13.878912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:13.926141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dc51-117b-40db-af50-92008b55c86f None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.926762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dc51-117b-40db-af50-92008b55c86f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.927258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dc51-117b-40db-af50-92008b55c86f None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.929613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 122/245] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:13.933838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fcdcf-9bc8-4506-8aa8-a4e415498657 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.934208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fcdcf-9bc8-4506-8aa8-a4e415498657 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.934698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fcdcf-9bc8-4506-8aa8-a4e415498657 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.936253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 123/246] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:13.943013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:13.943603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:13.944087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:13.961339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:13.962917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:13.994952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93165763-f394-4853-b8dd-15de81a512ce None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.011571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.035977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce129df-5d9d-4195-a2cd-dbf09d61a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.036476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.037483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce129df-5d9d-4195-a2cd-dbf09d61a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.037766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce129df-5d9d-4195-a2cd-dbf09d61a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.038042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce129df-5d9d-4195-a2cd-dbf09d61a9a3 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.039233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce129df-5d9d-4195-a2cd-dbf09d61a9a3 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:14.040988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce129df-5d9d-4195-a2cd-dbf09d61a9a3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:14.042681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 124/247] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:13 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:14.043210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 124/248] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:14.045033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.045033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.045033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.048887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.068699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:14.070295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:14.157694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 125/249] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:14.158845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.187948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764dc859-e80f-4a24-90b9-085292e1960a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.232667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.234219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.234842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.236483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.238054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] RBAC: Adding query filter params (role.id=3d876d008fdb464f93d245f9b7ef81c6, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=5c8ee13c6c4f4fe39bc3648bf2d4543d) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:14.239398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:14.243873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:14.262257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 125/250] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/role_assignments?user.id=5c8ee13c6c4f4fe39bc3648bf2d4543d&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=3d876d008fdb464f93d245f9b7ef81c6 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:14.263052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.318954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882557d-42b6-4025-9474-bb39692b5066 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.336586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.380217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.382436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.382731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.383015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.402934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.409638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:14.422049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:14.422978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:14.423800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 126/251] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:14.432488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.521670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a58c7-d4e8-4b89-af2a-e6ab47ece216 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.547134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891814-bf42-4a08-99ce-58ee46431a3d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.586526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.587737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.588189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.588612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.589536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:14.589988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:14.591599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:14.602329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 127/252] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:14.608564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.650391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.650391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.650391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.650391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.679726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.697358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:14.711516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:14.712698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:14.713833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 126/253] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:14.719449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.746771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f7c7-abb6-45da-adda-d368ee6c083c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.787727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.788627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.789022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.789308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] PATH_INFO: `/v3/users/50ec687901b94804bfa7acfab0abeadb` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.791602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] RBAC: Authorizing `identity:get_user(user_id=50ec687901b94804bfa7acfab0abeadb)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:14.793217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:14.794854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 128/254] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/users/50ec687901b94804bfa7acfab0abeadb => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:14.800345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.894717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e201-4754-4426-a9b9-7ae9adb7395d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.927074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.955077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.956385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.956678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.956779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b397cd-70db-49c1-8394-32aa5f48c92c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:14.957201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:14.957709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:14.958020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:14.959462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:14.959644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:14.978198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 127/255] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:14.983796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:14.997162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:14.997953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:14.998390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:14.998741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.078683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:15.080480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:15.155562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2edd92-b9d9-4ea9-8270-4920b3a854da None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:15.182791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:15.186388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 129/256] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:07:15.196397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:15.231189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:15.232445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.232931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.233845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] PATH_INFO: `/v3/users/4445867b60454175a1b82e451221cb86` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.235319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] RBAC: Authorizing `identity:get_user(user_id=4445867b60454175a1b82e451221cb86)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:15.237019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:15.238884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 128/257] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:14 2022] GET /identity/v3/users/4445867b60454175a1b82e451221cb86 => generated 297 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:15.249838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:15.348588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab212aa8-4fa6-4044-ba73-e7056d37c1a7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:15.357751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:15.384775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:15.385638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.385919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.386195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.387400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:15.391319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:15.391319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 130/258] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:15 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:15.396927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:15.458075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d78c0e-e77f-4686-bede-7f824b5fa985 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:15.468398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:15.512255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c8b83-0641-4a05-83c5-4afb51d6f622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:15.513131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c8b83-0641-4a05-83c5-4afb51d6f622 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.513411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c8b83-0641-4a05-83c5-4afb51d6f622 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.513684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c8b83-0641-4a05-83c5-4afb51d6f622 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.515570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c8b83-0641-4a05-83c5-4afb51d6f622 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:15.517050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c8b83-0641-4a05-83c5-4afb51d6f622 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:15.518952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 129/259] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:15 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:15.522172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801dd31-36ff-4ecb-99a9-971ec928879a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.522458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801dd31-36ff-4ecb-99a9-971ec928879a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.522738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801dd31-36ff-4ecb-99a9-971ec928879a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.527338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01860426-5771-47eb-ac6b-f7ebc957c427 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:15.529053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 130/260] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:15.530547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801dd31-36ff-4ecb-99a9-971ec928879a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:15.594767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:15.595955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.596470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.596943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.597990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=50ec687901b94804bfa7acfab0abeadb) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:15.598551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:15.600409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a7760-5f6f-4f0c-af59-addf4d5bcfaa None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:15.633946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 131/261] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:15 2022] GET /identity/v3/role_assignments?user.id=50ec687901b94804bfa7acfab0abeadb&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:15.637813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.638434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.638946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.659753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801dd31-36ff-4ecb-99a9-971ec928879a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:15.670532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:15.672114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:15.681475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:15.713064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:15.713980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:15.714261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:15.714537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/4445867b60454175a1b82e451221cb86/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:15.717780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] RBAC: Authorizing `identity:create_grant(project_id=a32ec861d2104c71ad90b203e9ae984c, user_id=4445867b60454175a1b82e451221cb86, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:15.719241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:15.779870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 132/262] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:15.818979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 131/263] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:07:15 2022] PUT /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/4445867b60454175a1b82e451221cb86/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:15.819526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:15.897740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:15.947217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff3f15-5b97-48cd-8d90-3c62552ab915 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:15.999489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:16.001158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:16.001654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:16.002128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:16.015390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:16.024191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:16.025153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:16.025932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:16.028846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 133/264] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:16.035148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:16.181899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a38e83-3683-4d2e-bc42-abd709dd4745 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:16.217266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:16.217944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:16.218231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:16.218507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:16.219258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:16.219592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:16.220966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:16.230933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 132/265] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:16.236539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:16.414532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe6e5-ca24-4fc9-ad0a-45ef2d8003b0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:16.444633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:16.462177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bf52c-c1ec-4588-8d52-33e38fad630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:16.463144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bf52c-c1ec-4588-8d52-33e38fad630a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:16.463460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bf52c-c1ec-4588-8d52-33e38fad630a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:16.463738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bf52c-c1ec-4588-8d52-33e38fad630a None admin] PATH_INFO: `/v3/users/50ec687901b94804bfa7acfab0abeadb` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:16.465098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bf52c-c1ec-4588-8d52-33e38fad630a None admin] RBAC: Authorizing `identity:get_user(user_id=50ec687901b94804bfa7acfab0abeadb)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:16.466626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bf52c-c1ec-4588-8d52-33e38fad630a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:16.468668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 134/266] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:16 2022] GET /identity/v3/users/50ec687901b94804bfa7acfab0abeadb => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:16.474500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:16.546791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:16.616841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28b0ca-7913-4c26-ad77-a6ae5d3034ce None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:16.678623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:16.679295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:16.679607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:16.679891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:16.791059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:16.792656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:16.841483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590235-84cc-4124-aab8-8529d387b9ed None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:16.841821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590235-84cc-4124-aab8-8529d387b9ed None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:16.842047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590235-84cc-4124-aab8-8529d387b9ed None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:16.843430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 135/267] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:16.848365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:16.848649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:16.848928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:16.850615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:16.884284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:16.885473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:16.910875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 133/268] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:16.918915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:16.977619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 136/269] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:17.007406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.106839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508b6b6-4563-49e3-b0cf-6d6a181af71f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.141002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a9632-ebd4-4dad-8e5c-c31c6e89bb87 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.186182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.187402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.187666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:17.187863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.188141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.189510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:17.192309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:17.194004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 134/270] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:16 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:17.200192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.218236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16acd63-6cb1-4914-9715-a77585ca0098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.219036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16acd63-6cb1-4914-9715-a77585ca0098 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.219332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16acd63-6cb1-4914-9715-a77585ca0098 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.219618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16acd63-6cb1-4914-9715-a77585ca0098 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.222576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16acd63-6cb1-4914-9715-a77585ca0098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:17.224066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16acd63-6cb1-4914-9715-a77585ca0098 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:17.225876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 137/271] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:17.227818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:17.229353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e48d9-d764-4d8d-afe6-2b0cd4745563 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.229828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e48d9-d764-4d8d-afe6-2b0cd4745563 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.230283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e48d9-d764-4d8d-afe6-2b0cd4745563 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.234457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 138/272] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:17.238377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.250407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.250407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.258882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:17.261107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:17.323902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154868a8-22f4-43fd-9acf-e27c718a5c4f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.349980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 139/273] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:17.355329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.380286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.380872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.381154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.381429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.382545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] RBAC: Adding query filter params (role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=50ec687901b94804bfa7acfab0abeadb) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:17.382855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:17.385843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:17.392612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 135/274] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/role_assignments?user.id=50ec687901b94804bfa7acfab0abeadb&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=a1af2eba782e475681e027a5d25335c6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:17.399333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.477423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5c78-e255-4844-9d9c-7ebf6742f79e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.497978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:17.518173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb802be3-9352-4f82-908b-29a09c242846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.519153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb802be3-9352-4f82-908b-29a09c242846 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.519667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb802be3-9352-4f82-908b-29a09c242846 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.520122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb802be3-9352-4f82-908b-29a09c242846 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.521169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb802be3-9352-4f82-908b-29a09c242846 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:17.522863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb802be3-9352-4f82-908b-29a09c242846 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:17.562204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:17.590954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f61d-5ddc-4180-8517-d8815132587b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.631967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.632802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.633125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.633408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.634848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 140/275] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:17 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:17.637532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b9471-5d1a-4376-968b-6e3ae8cffd10 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.638063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b9471-5d1a-4376-968b-6e3ae8cffd10 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.638555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b9471-5d1a-4376-968b-6e3ae8cffd10 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.640442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 141/276] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:17.649200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.649531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.649760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.660410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:17.669974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:17.671056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:17.671906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:17.674875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:17.675859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 136/277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:17.680627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:17.684540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.753629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 142/278] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:17.787603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.815426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fb7a5-41e6-4aa9-a0cd-03b5ac74a882 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.856536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.857564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.858131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.859810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:17.860322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:17.862074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:17.863904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:17.873230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 137/279] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:17.879141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:17.909730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1887d-129c-4304-86b4-2a790bf01167 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:17.912558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:17.951907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:17.953504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:17.954089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:17.954622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:17.967652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:17.977023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:17.977958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:17.978750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 143/280] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:17.984526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:18.005339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d066ab-3b1f-44dc-b5dd-04b6271cb7fa None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:18.048024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:18.048906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.049215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.049492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] PATH_INFO: `/v3/users/d4a5597ff2cb4bf8b1385f3a5f77a561` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.068529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] RBAC: Authorizing `identity:get_user(user_id=d4a5597ff2cb4bf8b1385f3a5f77a561)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:18.069986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:18.072049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 138/281] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/users/d4a5597ff2cb4bf8b1385f3a5f77a561 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:18.082091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:18.122958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7f7f1-445a-4174-ae6d-9e62d2fec4bb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:18.150904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:18.167789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:18.168399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.168682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.169108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.170005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:18.170311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:18.172694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:18.181877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 144/282] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:18.187834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:18.215467 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d0eec-cd89-47f6-974a-0a2430e5ce09 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:18.225646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:18.251584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:18.252068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.252907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.253182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.254011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, user.id=d4a5597ff2cb4bf8b1385f3a5f77a561, scope.system=all) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:18.254321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:18.255802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:18.262887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 139/283] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jun 1 04:07:18 2022] GET /identity/v3/role_assignments?user.id=d4a5597ff2cb4bf8b1385f3a5f77a561&scope.system=all&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:18.367615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879783-b4d5-4bf9-8fff-3a102b401afd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:18.422646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd414f06-3325-4123-833e-aa57958d5b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:18.423506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd414f06-3325-4123-833e-aa57958d5b69 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.423794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd414f06-3325-4123-833e-aa57958d5b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.424606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd414f06-3325-4123-833e-aa57958d5b69 None admin] PATH_INFO: `/v3/users/5c8ee13c6c4f4fe39bc3648bf2d4543d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.432322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd414f06-3325-4123-833e-aa57958d5b69 None admin] RBAC: Authorizing `identity:get_user(user_id=5c8ee13c6c4f4fe39bc3648bf2d4543d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:18.433846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd414f06-3325-4123-833e-aa57958d5b69 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:18.436069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 145/284] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:18 2022] GET /identity/v3/users/5c8ee13c6c4f4fe39bc3648bf2d4543d => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:18.441992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:18.641184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0785ec-1b09-4b97-b344-eb7ab85bfba1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:18.669744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:18.697435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d742f-369e-460f-a244-6ccffb0276d0 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.697435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d742f-369e-460f-a244-6ccffb0276d0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.697435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d742f-369e-460f-a244-6ccffb0276d0 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.697435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 146/285] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:18.697435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.697435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.697915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.697986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:18.697986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.697986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.697986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.699096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:18.700604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:18.702328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 140/286] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:18 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:18.706068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:18.707829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:18.707961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:18.830108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 147/287] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:18.858907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799834e9-f5a0-49fe-b166-7ea4eacd6f42 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:18.878593 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:18.913747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:18.923997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:18.926113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:18.926113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:18.926113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/5c8ee13c6c4f4fe39bc3648bf2d4543d/roles/3d876d008fdb464f93d245f9b7ef81c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:18.932067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a32ec861d2104c71ad90b203e9ae984c, user_id=5c8ee13c6c4f4fe39bc3648bf2d4543d, role_id=3d876d008fdb464f93d245f9b7ef81c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:18.933519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:19.009328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd60e1c-b82b-48ed-b726-bc4268c950e7 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:19.029616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 141/288] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:07:18 2022] PUT /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c/users/5c8ee13c6c4f4fe39bc3648bf2d4543d/roles/3d876d008fdb464f93d245f9b7ef81c6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:07:19.058574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:19.059412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:19.059701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:19.059987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:19.077380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:19.085440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:19.086317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:19.088438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 148/289] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:19.096538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:19.163729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:19.240410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1ab6d-9190-4d2c-9608-f2e2e597bbe3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:19.303087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:19.303810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:19.304095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:19.304368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:19.305134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:19.305438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:19.306924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:19.319576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 142/290] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:19.325021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:19.428666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:19.446902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec078d-de2f-4438-94b4-c77ef3acae2b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:19.492565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:19.493447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:19.493731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:19.494005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] PATH_INFO: `/v3/users/4445867b60454175a1b82e451221cb86` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:19.495221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] RBAC: Authorizing `identity:get_user(user_id=4445867b60454175a1b82e451221cb86)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:19.496725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:19.498704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 149/291] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:19 2022] GET /identity/v3/users/4445867b60454175a1b82e451221cb86 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:19.504130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:19.719091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e75d0-b964-4d96-beb2-571035f7a839 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:19.733904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:19.767445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:19.768779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:19.769066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:19.769345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:19.857408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:19.858892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:19.984730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 143/292] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:19.990783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:20.142697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc34ee-c7f4-4919-9c37-9eddf48a545a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:20.193627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b05ff8-a239-4fbb-9607-7e83a288857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:20.194542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b05ff8-a239-4fbb-9607-7e83a288857d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.194829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b05ff8-a239-4fbb-9607-7e83a288857d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.195112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b05ff8-a239-4fbb-9607-7e83a288857d None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:20.199326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b05ff8-a239-4fbb-9607-7e83a288857d None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:20.200862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b05ff8-a239-4fbb-9607-7e83a288857d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:20.202779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 150/293] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:19 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:20.208393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:20.221188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:20.344709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b80fc-6da4-46eb-a4e1-8dfa11bd4131 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:20.380267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:20.380930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.381216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.381742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:20.382586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=4445867b60454175a1b82e451221cb86) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:20.382905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:20.384390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5091e-987a-4af6-a3cf-e90701c6ea43 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:20.390894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 144/294] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:20 2022] GET /identity/v3/role_assignments?user.id=4445867b60454175a1b82e451221cb86&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:20.685374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbcb6a-85a9-48b4-9999-bf447939b54c None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.686328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbcb6a-85a9-48b4-9999-bf447939b54c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.686830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbcb6a-85a9-48b4-9999-bf447939b54c None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:20.688799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 151/295] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:20.694129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e5713-f7ca-4590-974f-d1e0dcf8b5d6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.694426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e5713-f7ca-4590-974f-d1e0dcf8b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.694711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e5713-f7ca-4590-974f-d1e0dcf8b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:20.712768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-508e5713-f7ca-4590-974f-d1e0dcf8b5d6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:20.713962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-508e5713-f7ca-4590-974f-d1e0dcf8b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:20.790883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 145/296] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:20.792142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066473a1-7aad-4349-9d06-f67be3796d28 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.792644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066473a1-7aad-4349-9d06-f67be3796d28 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.792926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066473a1-7aad-4349-9d06-f67be3796d28 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:20.794384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 152/297] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:20.800984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.800984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.800984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:20.805745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:20.817727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066473a1-7aad-4349-9d06-f67be3796d28 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:20.821078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:20.821078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:20.854464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:20.904267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 146/298] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:20.927621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:20.957748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066473a1-7aad-4349-9d06-f67be3796d28 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:20.993074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:20.993872 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:20.994321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:20.994750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.006109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:21.014281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:21.015384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:21.016387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:21.019135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 153/299] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:21.025726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.049533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc2c14-c274-4a46-be76-9cfc84f48c78 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.088252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:21.094056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.095060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.095542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.096039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.097368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:21.099040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:21.105576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 147/300] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:21.112792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.113114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:21.133787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301acb9-8c32-417e-a78b-649343d51a8d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.169743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.170738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.171230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.171610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.172356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:21.172721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:21.174194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2e050-ea3f-4ca6-9ffc-4f23f5044767 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:21.183347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 154/301] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:21.183885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75573ea-208c-4ff3-88d4-5cf7dc4052b3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.184087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75573ea-208c-4ff3-88d4-5cf7dc4052b3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.184363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75573ea-208c-4ff3-88d4-5cf7dc4052b3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.185727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 155/302] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:21.191008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75573ea-208c-4ff3-88d4-5cf7dc4052b3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.221534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbba2f-2920-4b92-8cf2-58b76017b1ce None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.259373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c67fc-1d3b-4a2c-b89f-ddb300511345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.260054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c67fc-1d3b-4a2c-b89f-ddb300511345 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.260339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c67fc-1d3b-4a2c-b89f-ddb300511345 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.260619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c67fc-1d3b-4a2c-b89f-ddb300511345 None admin] PATH_INFO: `/v3/groups` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.261332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c67fc-1d3b-4a2c-b89f-ddb300511345 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:21.262852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c67fc-1d3b-4a2c-b89f-ddb300511345 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:21.295048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75573ea-208c-4ff3-88d4-5cf7dc4052b3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.329421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.330948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.331245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.331555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] PATH_INFO: `/v3/users/0993f34c6b63462390272dc7fd747133` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.347326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 148/303] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Jun 1 04:07:21 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:21.347326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.347326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.347326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.350723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] RBAC: Authorizing `identity:get_user(user_id=0993f34c6b63462390272dc7fd747133)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:21.350985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:21.351510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:21.353530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 156/304] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/users/0993f34c6b63462390272dc7fd747133 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:21.359633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.368023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:21.369265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:21.439027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:21.459837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 149/305] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:21.476703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.480700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ae664-040f-49ab-b09a-8e5a0768264a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.519354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.523028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.523028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.523028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.523028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] RBAC: Adding query filter params (role.id=3d876d008fdb464f93d245f9b7ef81c6, user.id=0993f34c6b63462390272dc7fd747133, scope.system=all) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:21.523028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:21.523028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:21.535258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 157/306] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/role_assignments?user.id=0993f34c6b63462390272dc7fd747133&scope.system=all&role.id=3d876d008fdb464f93d245f9b7ef81c6 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:21.601085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94d347-528a-4f08-997d-da3d28b8aad2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.644007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492f008-67b0-453a-89ed-c2570c631004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.645153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492f008-67b0-453a-89ed-c2570c631004 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.645435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492f008-67b0-453a-89ed-c2570c631004 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.645710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492f008-67b0-453a-89ed-c2570c631004 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.659476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492f008-67b0-453a-89ed-c2570c631004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-4492f008-67b0-453a-89ed-c2570c631004 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:21.667562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:21.668492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:21.669280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 150/307] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:21.675658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.746349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:21.756344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bbd3b-b95b-4fe0-b313-7e98627e7a43 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.757003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bbd3b-b95b-4fe0-b313-7e98627e7a43 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.757234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bbd3b-b95b-4fe0-b313-7e98627e7a43 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.758597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 151/308] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:21.763862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.764157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.764433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.784142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:21.785543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:21.792443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338610-e32d-406b-882a-e8398b190e3a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:21.836945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:21.841380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:21.842586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:21.843022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:21.843521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:21.844568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:21.845488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:21.847769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:21.859130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 152/309] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:21.860552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 158/310] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:21.868524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.887693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:21.987330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f42e04-161c-4bc9-aef1-c8c480e29a7e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.017471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.018950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.019210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69882c-ce7c-40ff-9701-264341e76084 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.019432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.019568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] PATH_INFO: `/v3/users/d4a5597ff2cb4bf8b1385f3a5f77a561` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.020823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] RBAC: Authorizing `identity:get_user(user_id=d4a5597ff2cb4bf8b1385f3a5f77a561)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:22.022289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:22.024264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 153/311] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/users/d4a5597ff2cb4bf8b1385f3a5f77a561 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:22.032207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:22.051141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:22.061019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.062101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.062618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.065845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.075133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:22.084367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:22.085483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:22.086343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:22.087262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:22.094288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 159/312] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:22.096086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:22.139322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820dea9-febd-4c25-918b-6847e3b28c40 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.175056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.175964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.176253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.176534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] PATH_INFO: `/v3/system/users/d4a5597ff2cb4bf8b1385f3a5f77a561/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.178026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d4a5597ff2cb4bf8b1385f3a5f77a561, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:22.179683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:22.205767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43efa1-a436-4af7-a5a6-54c359988a1d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.241633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.242300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.242577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.242845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.243614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:22.245567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:22.245567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:22.254425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 160/313] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:22.259834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:22.265095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 154/314] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Jun 1 04:07:22 2022] PUT /identity/v3/system/users/d4a5597ff2cb4bf8b1385f3a5f77a561/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:07:22.324966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:22.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6adb-fe04-4264-a3c4-f8b679469a6b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.427199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.428320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.428714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.429117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] PATH_INFO: `/v3/users/5c8ee13c6c4f4fe39bc3648bf2d4543d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.430500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] RBAC: Authorizing `identity:get_user(user_id=5c8ee13c6c4f4fe39bc3648bf2d4543d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:22.432213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:22.434312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 161/315] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:22 2022] GET /identity/v3/users/5c8ee13c6c4f4fe39bc3648bf2d4543d => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:22.440164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:22.544137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69855f00-8eff-41b4-a0b0-2cb1731e90a8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.557553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:22.578333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.578908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.579194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.579502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.650142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:22.651673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:22.759636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 155/316] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:22.761715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:22.869206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574d58d-4d2c-45eb-b47a-205ef3444544 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:22.886119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:22.903993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95123ba5-e9ca-48af-a87b-dba1efc2d551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:22.904861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95123ba5-e9ca-48af-a87b-dba1efc2d551 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:22.905148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95123ba5-e9ca-48af-a87b-dba1efc2d551 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:22.905430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95123ba5-e9ca-48af-a87b-dba1efc2d551 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:22.906608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95123ba5-e9ca-48af-a87b-dba1efc2d551 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:22.908105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95123ba5-e9ca-48af-a87b-dba1efc2d551 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:22.910634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 162/317] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:22 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:22.916361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:22.931878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:23.017441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb710d5-48c1-4c72-bf6f-9f2f85afd9c9 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:23.052498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:23.053495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:23.053972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:23.054441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:23.055518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] RBAC: Adding query filter params (role.id=3d876d008fdb464f93d245f9b7ef81c6, scope.project.id=a32ec861d2104c71ad90b203e9ae984c, user.id=5c8ee13c6c4f4fe39bc3648bf2d4543d) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:23.056023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:23.057659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045beede-b581-404f-aa94-63c8c7a7d092 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:23.064466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 156/318] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:22 2022] GET /identity/v3/role_assignments?user.id=5c8ee13c6c4f4fe39bc3648bf2d4543d&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=3d876d008fdb464f93d245f9b7ef81c6 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:23.803015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7894a7d-a4c9-4b73-b986-b1927bfc8812 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:23.803598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7894a7d-a4c9-4b73-b986-b1927bfc8812 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:23.803598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7894a7d-a4c9-4b73-b986-b1927bfc8812 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:23.805013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 163/319] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:23.810120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:23.810419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:23.810704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:23.827507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:23.828708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:23.833036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4318-ec2c-420a-bc04-7f9dc41c9ae5 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:23.833325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4318-ec2c-420a-bc04-7f9dc41c9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:23.833602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4318-ec2c-420a-bc04-7f9dc41c9ae5 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:23.834983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 164/320] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:23.840883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:23.841349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:23.841643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:23.859932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:23.861123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:23.901202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 157/321] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:23.923628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:23.931965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:23.943329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 165/322] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:23.958615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:24.022030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce2839-86da-43ea-b08e-28061cd6d863 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.058205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.059280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.059897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.060325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.061480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:24.062922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:24.063027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904c774-e765-4943-8aa7-4135f6c847dd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.069821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 158/323] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:24.070556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:24.083286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:24.103234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.104070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.104348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.104631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.115761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:24.123484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:24.124361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:24.125137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 166/324] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:24.131180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:24.152820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:24.179028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358c74-c519-4de2-8c54-ab57e93d2a52 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.214936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.215641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.215932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.216422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] PATH_INFO: `/v3/groups` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.216902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:24.218355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:24.239819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a93fe5-65dc-4235-a24b-a7f148207784 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.278139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.278810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.279093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.279644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.280240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:24.280549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:24.281950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:24.290474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 167/325] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:24.296187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:24.312737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 159/326] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Jun 1 04:07:24 2022] POST /identity/v3/groups => generated 226 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:07:24.335845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:24.408346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500aa47-67cc-4a86-ac21-e8fe736b970f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.445028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0c993-9576-4bea-bef6-6f8e54824420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.445951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0c993-9576-4bea-bef6-6f8e54824420 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.446235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0c993-9576-4bea-bef6-6f8e54824420 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.446509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0c993-9576-4bea-bef6-6f8e54824420 None admin] PATH_INFO: `/v3/users/0993f34c6b63462390272dc7fd747133` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.447946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0c993-9576-4bea-bef6-6f8e54824420 None admin] RBAC: Authorizing `identity:get_user(user_id=0993f34c6b63462390272dc7fd747133)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:24.449415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0c993-9576-4bea-bef6-6f8e54824420 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:24.451261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 168/327] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:24 2022] GET /identity/v3/users/0993f34c6b63462390272dc7fd747133 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:24.457059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:24.561007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dd45c-41f3-445e-9c42-61d273e238da None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.585013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efc655-e023-476e-a91e-a585ea4969fc None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.585299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efc655-e023-476e-a91e-a585ea4969fc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.585635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efc655-e023-476e-a91e-a585ea4969fc None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.586974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 169/328] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:24.592165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.592515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.592798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.596466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.597335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.597613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.597892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] PATH_INFO: `/v3/system/users/0993f34c6b63462390272dc7fd747133/roles/3d876d008fdb464f93d245f9b7ef81c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.599399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0993f34c6b63462390272dc7fd747133, role_id=3d876d008fdb464f93d245f9b7ef81c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:24.600786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:24.602335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:24.613235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:24.614937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:24.690658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 170/329] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:24.702124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 160/330] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Jun 1 04:07:24 2022] PUT /identity/v3/system/users/0993f34c6b63462390272dc7fd747133/roles/3d876d008fdb464f93d245f9b7ef81c6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:24.714064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:24.756318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:24.842465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcadca-8d2a-48ba-a9e3-56c6670ce140 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:24.883563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:24.884376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:24.884656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:24.884931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:24.898343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:24.906402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:24.907565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:24.908984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 171/331] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:24.914702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:25.029698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b2ba2-f170-4d04-a796-5f64bb5ae811 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:25.067607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:25.068284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:25.068568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:25.068841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:25.069591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:25.069897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:25.071267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:25.080570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 161/332] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:25.085558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:25.209028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:25.216283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414beff1-4e48-401e-9c84-365ec4ffb536 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:25.250794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50df432-1274-4e78-abd3-fe1d61996937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:25.251745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50df432-1274-4e78-abd3-fe1d61996937 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:25.252066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50df432-1274-4e78-abd3-fe1d61996937 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:25.252342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50df432-1274-4e78-abd3-fe1d61996937 None admin] PATH_INFO: `/v3/users/d4a5597ff2cb4bf8b1385f3a5f77a561` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:25.253631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50df432-1274-4e78-abd3-fe1d61996937 None admin] RBAC: Authorizing `identity:get_user(user_id=d4a5597ff2cb4bf8b1385f3a5f77a561)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:25.255211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50df432-1274-4e78-abd3-fe1d61996937 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:25.257435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 172/333] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:25 2022] GET /identity/v3/users/d4a5597ff2cb4bf8b1385f3a5f77a561 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:25.263145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:25.332726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:25.383714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adfbad-bbc3-4e41-8b54-3abb072eee79 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:25.423148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:25.423855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:25.424142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:25.424416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:25.425248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] RBAC: Adding query filter params (role.id=b79d035ffc764400a134a4adcda8b7f8, user.id=d4a5597ff2cb4bf8b1385f3a5f77a561, scope.system=all) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:25.425556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:25.426984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5bdc2-e8fa-4865-99fc-f06ab884fe13 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:25.441921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 162/334] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jun 1 04:07:25 2022] GET /identity/v3/role_assignments?user.id=d4a5597ff2cb4bf8b1385f3a5f77a561&scope.system=all&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:27.204348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d7afa-6c91-4091-8549-13597da7f863 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:27.204913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d7afa-6c91-4091-8549-13597da7f863 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:27.204913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d7afa-6c91-4091-8549-13597da7f863 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:27.206561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 173/335] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:27.216710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:27.217222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:27.217422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:27.251462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:27.252704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:27.337511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 163/336] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:27.379170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d7afa-6c91-4091-8549-13597da7f863 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:27.521380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d7afa-6c91-4091-8549-13597da7f863 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:27.566707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:27.567564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:27.567852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:27.568129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:27.588334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:27.596793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:27.597745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:27.599155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 174/337] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:27.603802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:27.717673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:27.796557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd878a-2187-4277-8cbd-eaf8072eb9af None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:27.848929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:27.849619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:27.849909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:27.850194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:27.850959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:27.851269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:27.852685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:27.864622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 164/338] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:27.870128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:27.921215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:28.028317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b1db-3ee9-429a-891e-d38fb692e7f8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:28.067971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacc298-3582-452c-b81b-d5d1c8b9a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:28.068556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacc298-3582-452c-b81b-d5d1c8b9a92a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.068905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacc298-3582-452c-b81b-d5d1c8b9a92a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.069185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacc298-3582-452c-b81b-d5d1c8b9a92a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.150770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacc298-3582-452c-b81b-d5d1c8b9a92a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:28.152286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacc298-3582-452c-b81b-d5d1c8b9a92a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:28.186377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:28.271898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 175/339] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:28.277560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:28.278088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38002c-9144-463f-ac19-4613d1c38a32 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.278345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38002c-9144-463f-ac19-4613d1c38a32 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.278626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38002c-9144-463f-ac19-4613d1c38a32 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.279996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 176/340] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:07:28.285006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e8b8-923a-410f-a93a-14ad0be8173a None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.285300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e8b8-923a-410f-a93a-14ad0be8173a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.285587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0e8b8-923a-410f-a93a-14ad0be8173a None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.286999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 177/341] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:28.288088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d7a64-d12c-4508-8e3e-9ce7d4b22957 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.288376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d7a64-d12c-4508-8e3e-9ce7d4b22957 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.288655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d7a64-d12c-4508-8e3e-9ce7d4b22957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.309258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c69d7a64-d12c-4508-8e3e-9ce7d4b22957 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:28.310752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c69d7a64-d12c-4508-8e3e-9ce7d4b22957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:28.413472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 178/342] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:28.415084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48c1e-f8de-4763-95bc-d37d12208fa4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.415647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48c1e-f8de-4763-95bc-d37d12208fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.416121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d48c1e-f8de-4763-95bc-d37d12208fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.447345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-57d48c1e-f8de-4763-95bc-d37d12208fa4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:28.447859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d28e60-dbc0-44eb-804c-21cec8c73e57 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:28.448552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-57d48c1e-f8de-4763-95bc-d37d12208fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:28.459121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:28.492960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d3299-5bed-4584-bdd7-5fcc6dc48e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:28.493854 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3299-5bed-4584-bdd7-5fcc6dc48e91 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.494148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3299-5bed-4584-bdd7-5fcc6dc48e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.494432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3299-5bed-4584-bdd7-5fcc6dc48e91 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.495738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d3299-5bed-4584-bdd7-5fcc6dc48e91 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:28.497196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d3299-5bed-4584-bdd7-5fcc6dc48e91 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:28.498909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 165/343] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:28 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:28.500360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a3a7d-a9bf-455c-b7f8-2cd7024bb616 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.500648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a3a7d-a9bf-455c-b7f8-2cd7024bb616 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.501112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a3a7d-a9bf-455c-b7f8-2cd7024bb616 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.502935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 166/344] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:28.503755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a3a7d-a9bf-455c-b7f8-2cd7024bb616 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:28.515713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:28.548477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 179/345] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:28.550534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65676d77-4c9e-4816-b58c-1831c72bb328 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.550925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65676d77-4c9e-4816-b58c-1831c72bb328 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.551324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65676d77-4c9e-4816-b58c-1831c72bb328 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.556657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 180/346] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:28.556657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65676d77-4c9e-4816-b58c-1831c72bb328 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:28.636980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a3a7d-a9bf-455c-b7f8-2cd7024bb616 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:28.682006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85180-92df-44cc-8037-90c30cd35b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:28.682748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85180-92df-44cc-8037-90c30cd35b92 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.683034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85180-92df-44cc-8037-90c30cd35b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.683341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85180-92df-44cc-8037-90c30cd35b92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.685573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65676d77-4c9e-4816-b58c-1831c72bb328 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:28.695710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b85180-92df-44cc-8037-90c30cd35b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-48b85180-92df-44cc-8037-90c30cd35b92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:28.704526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:28.705469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:28.706260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 167/347] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:28.707702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.707702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.707702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.726418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:28.727631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:28.728251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0257b-2d73-4558-9479-4c3e0eb40f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:28.729214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0257b-2d73-4558-9479-4c3e0eb40f0b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.729505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0257b-2d73-4558-9479-4c3e0eb40f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.729779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0257b-2d73-4558-9479-4c3e0eb40f0b None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.741227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0257b-2d73-4558-9479-4c3e0eb40f0b None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:28.742951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0257b-2d73-4558-9479-4c3e0eb40f0b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:28.744877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 181/348] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:28 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:28.747293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.747809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.748149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:28.763513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:28.766580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:28.767838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:28.810364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 168/349] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:28.811053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:28.829723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:28.866190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 182/350] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:28.867062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:28.943883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea68c0f-20a9-468c-bc10-fcc53eec6239 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:28.990662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9fff0-064b-4a5a-9182-e23e9c48a577 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:28.992114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:28.993194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:28.993661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:28.994146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.007618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:29.015983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:29.016889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:29.018244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 169/351] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:29.018910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.039430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.040405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.040889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.041362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.042279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:29.042496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.045073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.054649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 183/352] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:29.055757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.070323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.103239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.164026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28503b4-4f4d-47b9-b419-911d79452a96 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.188145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda477-f217-4795-ad13-db4b39aae673 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.217360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.218172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.218627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.219190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.220599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=105f85dabeb0467da9127b26bd3c11cf) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:29.221139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.223532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.230116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.230911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 170/353] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:29.234849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.235027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.235646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.236069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.257323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:29.266394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:29.267662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:29.268499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:29.270465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 184/354] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:29.271439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.367557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.377793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665545c8-6068-4add-bcb9-477260daf27a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.399667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.404630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590b7fa-ecb0-4f7d-896e-351bddb808e0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.423146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.424926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.425401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.425904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.440777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:29.451963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:29.452930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:29.453750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:29.457898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.458069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 171/355] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:29.458158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.458158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.458295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.458377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.458377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:29.458377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.458859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.470442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 185/356] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:29.471743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.591950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be7b8b-5a27-440f-86d5-6905cd3c970c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.607365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4a4c9-95a0-48a7-9f22-ca1c5f9946ae None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.641840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.642765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.643047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.643354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] PATH_INFO: `/v3/users/0993f34c6b63462390272dc7fd747133` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.644618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] RBAC: Authorizing `identity:get_user(user_id=0993f34c6b63462390272dc7fd747133)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.646096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.651378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 172/357] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/users/0993f34c6b63462390272dc7fd747133 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:29.651378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.651606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.653286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.653468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.653821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.655689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:29.655689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.656883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.668824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 186/358] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:29.669596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.677968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.701590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.780077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3c7f-013c-40ba-a15d-962de9bc6124 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.799857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7dcdd2-359f-49b2-bf18-ddbbd3c47edd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:29.824635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.826000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.826290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.826570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.827735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:29.828332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.830696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b9cd7-3fa8-40bc-8a2c-20327e29776f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.840357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 173/359] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:29.843514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:29.844417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.844933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.845219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.846516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5982d81-2f95-43cd-885c-c8dc87418349 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:29.846818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5982d81-2f95-43cd-885c-c8dc87418349 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:29.847104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5982d81-2f95-43cd-885c-c8dc87418349 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:29.848505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 174/360] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:29.849612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5982d81-2f95-43cd-885c-c8dc87418349 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:29.932467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:29.935343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:29.960261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.964377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:29.990399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5982d81-2f95-43cd-885c-c8dc87418349 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.037635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.038337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.038796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.039142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.040640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] RBAC: Adding query filter params (role.id=3d876d008fdb464f93d245f9b7ef81c6, user.id=0993f34c6b63462390272dc7fd747133, scope.system=all) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:30.040640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:30.041910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:30.056312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 175/361] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/role_assignments?user.id=0993f34c6b63462390272dc7fd747133&scope.system=all&role.id=3d876d008fdb464f93d245f9b7ef81c6 => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:30.057374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.086512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 187/362] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:30.087667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.198351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc493-ff99-4c87-b3bf-de9d03e74075 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.233117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b99ed-5787-4417-866f-bb41a8d4b5f0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.243622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b4857-c250-45c1-b4f1-adace92e9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.244393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b4857-c250-45c1-b4f1-adace92e9da1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.244744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b4857-c250-45c1-b4f1-adace92e9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.245093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b4857-c250-45c1-b4f1-adace92e9da1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.257753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:30.262030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:30.277252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.278223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.278721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.279206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.326647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b4857-c250-45c1-b4f1-adace92e9da1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:30.328475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b4857-c250-45c1-b4f1-adace92e9da1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:30.364581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:30.366063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:30.456638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 176/363] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:30.459591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.459591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.459591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.477034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:30.478462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:30.495531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 188/364] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:30.497156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.523257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:30.532601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:30.571819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 177/365] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:30.573152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.630732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cb92-dc46-4b59-9516-250425812309 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.679004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.680449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.681255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.681751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.683667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:30.685966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:30.686996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdb294-9423-4db0-9d1a-69efda664d39 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.688600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 189/366] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:30.695186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.732123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.733102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.733390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.733764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.734940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:30.736957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:30.738856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 178/367] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:30.739763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.781142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:30.814318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:30.826402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2463002-cdb0-4a68-a130-71879770fa43 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.855477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e586b22-91df-4e31-9556-50ad723e0802 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:30.865644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.866695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.867096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.867661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.868830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:30.870445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:30.872344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 190/368] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:30.873116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:30.895135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:30.895924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:30.896183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:30.896459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:30.915321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:30.922554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:30.923495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:30.924884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 179/369] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:30.926158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.003084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e152b6-ab7d-49b1-b941-6e7f5c776108 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.039709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:31.043728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a508dd6-ab8b-460a-b623-96ecbd525271 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.048712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.048712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.048712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.048712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.049592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.050441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.052885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 191/370] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.054021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.085728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.086607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.086893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.087168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.088408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.089882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.091574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 180/371] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:30 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.097767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.098794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:31.198640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f07e8-51db-449d-b1f1-63b055b8b5c3 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.226994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ddc4d-b830-4138-bd78-86e1a55ee72c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.243146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.244450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.245020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.245310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.249406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.249545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.253343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 192/372] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.253343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.268512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.269186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.269469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.269742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.271336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:31.271336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.272169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.284370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 181/373] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.285223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.323975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:31.375380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:31.383139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7dd8e-7ca0-4f86-96de-7f5f8c4118ad None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.404419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8370b-90a7-4892-b614-0b423cf5bbbe None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.420633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.421292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.421605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.421986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.422817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=105f85dabeb0467da9127b26bd3c11cf) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:31.423122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.424653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.430183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 193/374] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=a1af2eba782e475681e027a5d25335c6 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.430960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.459041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.460462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.460915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.461631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.463216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=a32ec861d2104c71ad90b203e9ae984c) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:31.463733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.465418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.475464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 182/375] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.476674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.627500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a00df-3af4-4d32-b886-1f64e493cd8f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.670889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:31.686859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055e807-078a-4b4d-b838-5e902897803b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:31.690442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.691182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.691503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.691789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.692640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=a32ec861d2104c71ad90b203e9ae984c) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:31.692955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.694426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8739b7-67d1-4721-adc0-33268eadad95 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:31.704088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 194/376] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=a1af2eba782e475681e027a5d25335c6 => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:31.706299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb2da9-fdef-4947-89e9-378d6d40f7f3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.706592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb2da9-fdef-4947-89e9-378d6d40f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.706870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb2da9-fdef-4947-89e9-378d6d40f7f3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.709394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 195/377] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:31.722395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.722835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.723218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.742853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:31.744373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:31.765847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:31.766942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:31.768233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:31.768525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:31.768805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:31.835105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 196/378] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:31.871707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:31.910320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:31.911872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.042352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78e8c-d35d-4342-831c-817d12bf1fe2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.066891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:32.088376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 183/379] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:32.096326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.117172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.118390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.118681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.118964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.143148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:32.158802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:32.159713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:32.160500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:32.165908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 197/380] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:32.169812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.187507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:32.278216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f21c0-de98-4c5d-aa0d-7d87b9f555a3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.297871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6af3-05ea-4b14-a18b-3a2f9010dfa9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.340700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.341952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.343614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.344118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] PATH_INFO: `/v3/projects/efd554ba71f4463abb4c78dc3cafe52a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.346851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] RBAC: Authorizing `identity:get_project(project_id=efd554ba71f4463abb4c78dc3cafe52a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:32.348552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.358466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 184/381] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:32 2022] GET /identity/v3/projects/efd554ba71f4463abb4c78dc3cafe52a => generated 344 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:32.359722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.372124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.373111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.373600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.374073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.375033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:32.375568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:32.377221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.391837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 198/382] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:32.396141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.473818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:32.525237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:32.539723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639342-b7ad-4717-97ef-321d57cedb24 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.557841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146ec1-5c7b-42a1-8bb6-430ccad0c2bf None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.592923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.593788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.594073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.594353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.596117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:32.597562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.604211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 199/383] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:32 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:32.605352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.624031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.625262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.625753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.626262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] PATH_INFO: `/v3/groups/5dec506f5c864d4fbbe3f31b8461b930` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.644218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] RBAC: Authorizing `identity:get_group(group_id=5dec506f5c864d4fbbe3f31b8461b930)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:32.645771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.648015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 185/384] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:32 2022] GET /identity/v3/groups/5dec506f5c864d4fbbe3f31b8461b930 => generated 229 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:32.656391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.772479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e0f70-bf04-4a22-8a9d-fd9b958f5cf2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.806454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61372-7457-4520-b26e-d6f056c8b1fc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:32.839029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.840391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.840885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.841364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.844203 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:32.846589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.851430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 200/385] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:32 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:32.851656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:32.856118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:32.871926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:32.872942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:32.873436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:32.874012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:32.874890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:32.875511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] RBAC: Adding query filter params (group.id=5dec506f5c864d4fbbe3f31b8461b930, role.id=02d9e6449ea143beb203824112640698, scope.project.id=efd554ba71f4463abb4c78dc3cafe52a) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:32.876117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:32.877810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b5b7e-e75f-4660-b61d-55dc5f3607ce None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:32.884370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 186/386] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:32 2022] GET /identity/v3/role_assignments?group.id=5dec506f5c864d4fbbe3f31b8461b930&scope.project.id=efd554ba71f4463abb4c78dc3cafe52a&role.id=02d9e6449ea143beb203824112640698 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:33.016909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748ac45-a068-49e1-a45b-92c7aded6ab0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:33.053248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c0a69-23bf-400f-8785-fccccb3cb0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:33.054128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c0a69-23bf-400f-8785-fccccb3cb0a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:33.054405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c0a69-23bf-400f-8785-fccccb3cb0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:33.054678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c0a69-23bf-400f-8785-fccccb3cb0a7 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:33.056663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c0a69-23bf-400f-8785-fccccb3cb0a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=105f85dabeb0467da9127b26bd3c11cf, group_id=3924ec10c9ff4c34ad8b9f9689823c0a, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:33.058127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c0a69-23bf-400f-8785-fccccb3cb0a7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:33.135508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 201/387] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jun 1 04:07:32 2022] PUT /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:34.034313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7d4dd-cfeb-4ad0-8b86-138b20162227 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.035375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7d4dd-cfeb-4ad0-8b86-138b20162227 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.035883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7d4dd-cfeb-4ad0-8b86-138b20162227 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.037732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 187/388] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:34.042728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593305af-5145-46ba-9d62-e17d554db284 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.043576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593305af-5145-46ba-9d62-e17d554db284 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.044074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593305af-5145-46ba-9d62-e17d554db284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.047402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d644683-0077-43e1-912e-a2c92e5119a4 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.048442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d644683-0077-43e1-912e-a2c92e5119a4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.049989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d644683-0077-43e1-912e-a2c92e5119a4 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.054256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 188/389] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:34.058464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.059023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.059551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.068158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-593305af-5145-46ba-9d62-e17d554db284 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:34.069733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-593305af-5145-46ba-9d62-e17d554db284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:34.091658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:34.092879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:34.172683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:34.182308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 202/390] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:34.205487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 189/391] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:34.218658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593305af-5145-46ba-9d62-e17d554db284 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.243902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.320634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593305af-5145-46ba-9d62-e17d554db284 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:34.372145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:34.373013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.373273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.373561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.398443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae78480-0397-4a63-9f63-61d482c6737b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:34.408582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:34.416601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:34.417575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:34.418403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:34.424156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 203/392] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:34.430564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.432908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:34.447734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:34.448917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.449437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.450437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.474527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:34.490895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:34.491824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:34.492596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:34.494685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 190/393] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:34.499236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.527653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:34.566582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0864468-0581-49b5-8af5-4e30344f6006 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:34.617266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:34.618251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.618764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.619235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.620207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:34.620839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:34.622443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:34.632976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 204/394] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:34.638766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.672539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4ac64-f28e-4ba3-a4f2-ca75d5dc1c41 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:34.713662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:34.714403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.714697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.714976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.716606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:34.716921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:34.718356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:34.739438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 191/395] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:34.746721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.803881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:34.845211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb6336-42d2-4ab1-9a2b-b1e14d17aa87 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:34.873209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:34.883372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29e716-fe42-4f64-99aa-28df9596b108 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:34.885764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d1ed9-8118-41ee-b231-3188ca8cd15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:34.890254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d1ed9-8118-41ee-b231-3188ca8cd15b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.890254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d1ed9-8118-41ee-b231-3188ca8cd15b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.890254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d1ed9-8118-41ee-b231-3188ca8cd15b None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.890254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d1ed9-8118-41ee-b231-3188ca8cd15b None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:34.890254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d1ed9-8118-41ee-b231-3188ca8cd15b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:34.892069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 205/396] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:34.893544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa738f-199a-4a75-aeb9-9d9f8da3a8a4 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.893832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa738f-199a-4a75-aeb9-9d9f8da3a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.894112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa738f-199a-4a75-aeb9-9d9f8da3a8a4 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.895665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 206/397] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:34.898036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa738f-199a-4a75-aeb9-9d9f8da3a8a4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:34.933104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fa3b7-eddd-48f5-b99f-3f61c841550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:34.934251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fa3b7-eddd-48f5-b99f-3f61c841550e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.934573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fa3b7-eddd-48f5-b99f-3f61c841550e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.935065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fa3b7-eddd-48f5-b99f-3f61c841550e None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.936965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fa3b7-eddd-48f5-b99f-3f61c841550e None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:34.938519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fa3b7-eddd-48f5-b99f-3f61c841550e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:34.940567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 192/398] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:34.941746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:34.942032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:34.942313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:34.964400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:34.966127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:35.024096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa738f-199a-4a75-aeb9-9d9f8da3a8a4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.055437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 193/399] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:35.056426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.067454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.068403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.068750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.069035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.070449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:35.072056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:35.073961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 207/400] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:34 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:35.080804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.131758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:35.161494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:35.174398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba56bc-7d4f-44d6-b4a3-49e64b9e178c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.213638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.214948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.215380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.215769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.217110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735c9a9-9c72-46c8-99e9-5d8dfa1e89e9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.217243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:35.219057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:35.227335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 194/401] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:35 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:35.230091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.264444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.265301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.265595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.266009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.280226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:35.291435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:35.292387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:35.293168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:35.301559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 208/402] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:35.302641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.351907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffbd2c-8515-4108-95da-88bcd0354162 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.359847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:35.389083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.389896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.390286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.390646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.392862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] RBAC: Authorizing `identity:create_grant(project_id=a32ec861d2104c71ad90b203e9ae984c, group_id=3924ec10c9ff4c34ad8b9f9689823c0a, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:35.395494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:35.413349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce3a48-3ddc-4f02-87d8-d345d89b6a4a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.430911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:35.447904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9e4b-84ee-4718-9a55-74dc56a835d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.448790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9e4b-84ee-4718-9a55-74dc56a835d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.449079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9e4b-84ee-4718-9a55-74dc56a835d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.449362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9e4b-84ee-4718-9a55-74dc56a835d6 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.451692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9e4b-84ee-4718-9a55-74dc56a835d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=105f85dabeb0467da9127b26bd3c11cf, group_id=3924ec10c9ff4c34ad8b9f9689823c0a, role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:35.453155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9e4b-84ee-4718-9a55-74dc56a835d6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:35.463283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 195/403] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jun 1 04:07:35 2022] PUT /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:35.464442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.564476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 209/404] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jun 1 04:07:35 2022] PUT /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/a1af2eba782e475681e027a5d25335c6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:35.568255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56daeb-cc4d-472a-ba37-6b58581588b3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.568732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56daeb-cc4d-472a-ba37-6b58581588b3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.569042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56daeb-cc4d-472a-ba37-6b58581588b3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.574116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 210/405] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:35.582179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.582539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.582750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.608695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d0165-578b-45fe-888b-1ebb5bde538c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.620438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:35.621754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:35.656090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.657314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.657606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.658438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.659231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:35.659642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:35.661409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:35.672499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 196/406] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:35.680481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.718006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:35.752816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 211/407] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:35.794268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.805690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193265c-de33-4b54-9c09-7b909642d0c5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:35.852739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:35.853642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:35.853930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:35.854215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:35.856617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:35.858140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:35.867428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 197/408] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:35 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:35.879334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:35.994517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d33954-0184-4c92-9413-1aef95a5296c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.017885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:36.043768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:36.056133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c128af-4fa3-441e-9dbb-a54ccb8c35b4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.060988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46015dc-feea-4d51-8180-dc086f718faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.061795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46015dc-feea-4d51-8180-dc086f718faf None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.062083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46015dc-feea-4d51-8180-dc086f718faf None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.062361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46015dc-feea-4d51-8180-dc086f718faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.083137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46015dc-feea-4d51-8180-dc086f718faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-a46015dc-feea-4d51-8180-dc086f718faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:36.090943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:36.091905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:36.092725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:36.093959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 212/409] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:36.095081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a566c2-241b-4342-bc06-448ed731cc53 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.095426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a566c2-241b-4342-bc06-448ed731cc53 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.095711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a566c2-241b-4342-bc06-448ed731cc53 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.096982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 213/410] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:36.100436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d55162-97aa-47a9-9bab-20bb232836ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.101956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55162-97aa-47a9-9bab-20bb232836ef None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.102445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55162-97aa-47a9-9bab-20bb232836ef None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.103053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55162-97aa-47a9-9bab-20bb232836ef None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.104189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a566c2-241b-4342-bc06-448ed731cc53 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.105083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d55162-97aa-47a9-9bab-20bb232836ef None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:36.106769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d55162-97aa-47a9-9bab-20bb232836ef None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:36.114751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 198/411] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:35 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:36.114751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.114751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.114751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.137339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:36.138524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:36.232935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 199/412] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:36.233833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.311179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a566c2-241b-4342-bc06-448ed731cc53 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.373130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.373847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.374135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.374413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.375165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:36.375496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:36.378318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:36.399720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:36.400647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 214/413] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:36.401437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.409500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563eec-9c87-4435-b7e5-deeb2e7b123b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.443854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:36.446465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.447282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.447594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.447869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.450064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] RBAC: Authorizing `identity:create_grant(project_id=a32ec861d2104c71ad90b203e9ae984c, group_id=3924ec10c9ff4c34ad8b9f9689823c0a, role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:36.451545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:36.519278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f4644-34a3-4d35-96f5-f660ec2ff0d2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.559367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.559367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.559367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.559367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.560128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 200/414] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jun 1 04:07:36 2022] PUT /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c/groups/3924ec10c9ff4c34ad8b9f9689823c0a/roles/a1af2eba782e475681e027a5d25335c6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:36.561668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.586207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:36.600416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:36.601370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:36.602155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 215/415] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:36.612833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.778810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285b63c-71b1-4010-9e5d-881bb4b30585 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.813692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:36.831478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b9885-9918-4cb1-8552-7160e5f3d082 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:36.840947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.842485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.842923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.843368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] PATH_INFO: `/v3/groups/5dec506f5c864d4fbbe3f31b8461b930` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.845308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] RBAC: Authorizing `identity:get_group(group_id=5dec506f5c864d4fbbe3f31b8461b930)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:36.847211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:36.849616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 201/416] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:36 2022] GET /identity/v3/groups/5dec506f5c864d4fbbe3f31b8461b930 => generated 229 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:36.854488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.887808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:36.893073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:36.893833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:36.894192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:36.894627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:36.895508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:36.901032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:36.901032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:36.907547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 216/417] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:36.913119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:36.991622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5baafe-4a10-4de5-afce-ec1cf681cbb7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:37.025099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070dbc2-8e23-4db0-bb24-edfb2f3500aa None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:37.041301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:37.042622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:37.043400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:37.043863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] PATH_INFO: `/v3/projects/efd554ba71f4463abb4c78dc3cafe52a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:37.045260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] RBAC: Authorizing `identity:get_project(project_id=efd554ba71f4463abb4c78dc3cafe52a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:37.047638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:37.050507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 202/418] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:36 2022] GET /identity/v3/projects/efd554ba71f4463abb4c78dc3cafe52a => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:37.056119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:37.076503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:37.077132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:37.077417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:37.077691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:37.116257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:37.213418 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:37.214216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:37.214922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:37.238795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f0902-1bd1-4d1c-a8a1-5241a787e9ec None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:37.316573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:37.317721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:37.318175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:37.318614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] PATH_INFO: `/v3/projects/efd554ba71f4463abb4c78dc3cafe52a/groups/5dec506f5c864d4fbbe3f31b8461b930/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:37.324621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=efd554ba71f4463abb4c78dc3cafe52a, group_id=5dec506f5c864d4fbbe3f31b8461b930, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:37.326315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:37.349993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 217/419] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:37.353962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:37.406551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:37.419420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 203/420] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jun 1 04:07:37 2022] PUT /identity/v3/projects/efd554ba71f4463abb4c78dc3cafe52a/groups/5dec506f5c864d4fbbe3f31b8461b930/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:07:37.475235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cc73c-d8d3-4f51-aa97-f1474b834ac3 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:37.513720 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:37.514615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:37.514906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:37.515187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:37.516552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:37.518656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:37.521165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 218/421] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:37 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:37.528460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:37.713167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062ab7-c8cf-431b-b95c-7311f32ac2a9 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:37.757585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92202f63-2faa-4a84-a3e0-5ddffa185d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:37.758787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92202f63-2faa-4a84-a3e0-5ddffa185d31 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:37.759294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92202f63-2faa-4a84-a3e0-5ddffa185d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:37.759824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92202f63-2faa-4a84-a3e0-5ddffa185d31 None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:37.761682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92202f63-2faa-4a84-a3e0-5ddffa185d31 None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:37.764021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92202f63-2faa-4a84-a3e0-5ddffa185d31 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:37.766146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 204/422] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:37 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:37.775345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:37.930579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ccd57-f129-4606-8ea6-b3097aec6d92 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:37.960282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:37.981884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:37.982826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:37.982826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:37.983079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:37.983957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=105f85dabeb0467da9127b26bd3c11cf) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:37.984278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:37.985721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd99ec6-805f-4b75-9e76-1ebc6632b079 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:37.994032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 219/423] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:37 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:38.165337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6adbd-9ced-47d3-a20e-b1e2a37368e8 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.166029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6adbd-9ced-47d3-a20e-b1e2a37368e8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.166520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6adbd-9ced-47d3-a20e-b1e2a37368e8 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.168498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 205/424] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:38.174998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.175991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.176469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.204654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:38.206227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:38.217174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfaacb-9452-4efb-876d-b2e086dcf198 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.217699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfaacb-9452-4efb-876d-b2e086dcf198 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.217774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfaacb-9452-4efb-876d-b2e086dcf198 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.219459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 206/425] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:38.225158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.225460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.225729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.244599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:38.245881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:38.263694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:38.319272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 220/426] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:38.333246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 207/427] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:38.344616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:38.357873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:38.468942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08e48-ff76-406f-8e96-bd5be5a087c6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:38.473552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdbba2-38a9-4942-bb68-dc69da966f70 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:38.491240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:38.509984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:38.510971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.511274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.511657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.515762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:38.516776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.517524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.518071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.523919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:38.531421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:38.532438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:38.533872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:38.534674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 208/428] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:38.540295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:38.543611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:38.544461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 01 04:07:38.545251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:38.549163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 221/429] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:38.556870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:38.559967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:38.675359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485be41e-4b47-4e0d-b158-ab5c50f400c6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:38.702083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ca052-f596-48c3-9bc0-5b0764c9abf6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:38.721465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:38.722218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.722520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.722804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.723909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:38.724230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:38.725629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:38.736061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 209/430] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:38.741702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:38.746786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:38.747521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.747871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.748155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.748967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:38.749290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:38.750686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:38.760308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 222/431] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:07:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:38.765956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:38.811687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:38.819201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:38.857066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044d6c3-2c11-48c6-9514-8fcbb30469bb None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:38.883265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dbd27-a501-47e5-888c-d4eea2c41f33 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:38.897325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:38.898162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.898456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.899139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.923508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:38.924165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:38.924451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:38.924729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:38.989546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:38.990980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.011528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.013765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.075442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.123447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 210/432] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:07:39.124007 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.125832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.159720 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 223/433] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:07:39.165889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.246325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207124-2cc2-4563-9e9c-c6e43664b874 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.271795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d492-47e1-4780-ba98-c5d58821206e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.285956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.287171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.287652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.288142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] PATH_INFO: `/v3/projects/105f85dabeb0467da9127b26bd3c11cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.289490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=105f85dabeb0467da9127b26bd3c11cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.291180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.293476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 211/434] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/projects/105f85dabeb0467da9127b26bd3c11cf => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:39.300108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.314424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.315549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.315950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.316329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.317737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.319571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.325920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 224/435] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:39.327449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.332134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.348466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.413820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136821-b134-4984-b881-b5460c574aa1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.435874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a537a-f96e-4163-8737-5f6996ae581d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.454095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.455379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.455912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.456468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.457887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.459642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.466249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 212/436] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:39.467405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.474065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead1c85-3d13-4476-9d59-24367276803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.475178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead1c85-3d13-4476-9d59-24367276803a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.475516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead1c85-3d13-4476-9d59-24367276803a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.475796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead1c85-3d13-4476-9d59-24367276803a None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.477019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead1c85-3d13-4476-9d59-24367276803a None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.478491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead1c85-3d13-4476-9d59-24367276803a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.480283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 225/437] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:39.484236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09279a1a-7e96-41f6-bae8-a5866bba4706 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.484743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09279a1a-7e96-41f6-bae8-a5866bba4706 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.485227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09279a1a-7e96-41f6-bae8-a5866bba4706 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.487175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 226/438] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:39.487784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09279a1a-7e96-41f6-bae8-a5866bba4706 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.571295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.573720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cac92-d924-481c-981b-006f6b0f924a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.581417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.603230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09279a1a-7e96-41f6-bae8-a5866bba4706 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.607579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.608245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.608533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.608815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.609659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=105f85dabeb0467da9127b26bd3c11cf) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:39.609971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.611991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3ce32-a960-495e-a9e3-35b12332541d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.625158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 213/439] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=105f85dabeb0467da9127b26bd3c11cf&role.id=a1af2eba782e475681e027a5d25335c6 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:39.625158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.625326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.625786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.646832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:39.648146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:39.652188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.652858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.653304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.653725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.654723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=b79d035ffc764400a134a4adcda8b7f8, scope.project.id=a32ec861d2104c71ad90b203e9ae984c) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:39.655191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:39.656891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:39.671851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 227/440] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=b79d035ffc764400a134a4adcda8b7f8 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:39.732065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 214/441] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:39.757868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:39.883739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc553c08-ba1a-4d59-bf73-6e47c56d7999 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:39.923192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4f71b-d383-4ea8-a115-dc6989e63d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:39.924075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4f71b-d383-4ea8-a115-dc6989e63d1e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:39.924359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4f71b-d383-4ea8-a115-dc6989e63d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:39.924632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4f71b-d383-4ea8-a115-dc6989e63d1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:39.927809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:39.937557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4f71b-d383-4ea8-a115-dc6989e63d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4be4f71b-d383-4ea8-a115-dc6989e63d1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:39.946537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:39.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:39.948454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 228/442] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:39.953639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.065194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02dcc8-aa82-4e61-9a6c-6186e6b2933c None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.065489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02dcc8-aa82-4e61-9a6c-6186e6b2933c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.065969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.066075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02dcc8-aa82-4e61-9a6c-6186e6b2933c None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.067633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 229/443] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:40.069097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8623798-1ff3-4ab3-b85d-4ce9e7727eef None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.072753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.073042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.073323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.094049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:40.095418 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:40.108995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.109681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.109969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.110251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.111049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:40.111422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.112812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.122679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 215/444] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:40.129000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.181510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 230/445] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:40.205438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.244624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c42764-6a49-4854-99eb-2078065119f2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.286152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.286805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.287096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.287403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.317366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff8657-8cbf-43c2-8797-57c4990809c2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.317810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.342612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.351048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.351879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.352158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.352431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.354851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.356346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.362959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:40.370653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:40.371742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:40.372617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 231/446] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:40.378064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.457243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 216/447] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:07:40.462601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.476276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c8b0c-962f-4d28-9119-b1d2ee9a7143 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.509853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.510511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.510795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.511071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.511843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:40.512150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.513528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.526783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.526927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 232/448] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:40.527616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.560914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb659fd-4705-46b5-9988-8e906e642c34 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.577052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.594210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.595073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.595384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.595664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] PATH_INFO: `/v3/projects/a32ec861d2104c71ad90b203e9ae984c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.596825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] RBAC: Authorizing `identity:get_project(project_id=a32ec861d2104c71ad90b203e9ae984c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.598255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.599969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 217/449] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/projects/a32ec861d2104c71ad90b203e9ae984c => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:40.605401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.628391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758c767-efc7-4e8f-8eee-dd0a01878bec None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.662608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.663248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.663563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.663844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.707432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729da9-4f92-4c22-955d-f0ae289ed85c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.730644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.732144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.734090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.739697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.740556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.740834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.741108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] PATH_INFO: `/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.742279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] RBAC: Authorizing `identity:get_group(group_id=3924ec10c9ff4c34ad8b9f9689823c0a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.743735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.745314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 218/450] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/groups/3924ec10c9ff4c34ad8b9f9689823c0a => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:40.751060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.780649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:40.831945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 233/451] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:07:40.837552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:40.852227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49866a9-c262-4f7b-bbbd-6ec85613afdc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.885873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.886526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.886813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.887093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.887964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] RBAC: Adding query filter params (group.id=3924ec10c9ff4c34ad8b9f9689823c0a, role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=a32ec861d2104c71ad90b203e9ae984c) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:40.888279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.889725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.895990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 219/452] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/role_assignments?group.id=3924ec10c9ff4c34ad8b9f9689823c0a&scope.project.id=a32ec861d2104c71ad90b203e9ae984c&role.id=a1af2eba782e475681e027a5d25335c6 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:40.936773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3e43a-b00d-4a1f-94af-d02d076fad3f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:40.973116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:40.973894 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:40.974175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:40.974448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] PATH_INFO: `/v3/projects/efd554ba71f4463abb4c78dc3cafe52a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:40.975702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] RBAC: Authorizing `identity:get_project(project_id=efd554ba71f4463abb4c78dc3cafe52a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:40.977188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:40.978950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 234/453] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/projects/efd554ba71f4463abb4c78dc3cafe52a => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:40.984736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:41.050222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:41.090013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b026f-04c9-41ba-97b9-3d3453fe0516 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:41.125147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6a8c6-c562-4aa6-bbee-156469a34004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:41.126019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6a8c6-c562-4aa6-bbee-156469a34004 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:41.126304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6a8c6-c562-4aa6-bbee-156469a34004 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:41.127145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6a8c6-c562-4aa6-bbee-156469a34004 None admin] PATH_INFO: `/v3/groups/5dec506f5c864d4fbbe3f31b8461b930` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:41.128371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6a8c6-c562-4aa6-bbee-156469a34004 None admin] RBAC: Authorizing `identity:get_group(group_id=5dec506f5c864d4fbbe3f31b8461b930)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:41.129817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6a8c6-c562-4aa6-bbee-156469a34004 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:41.131718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 220/454] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jun 1 04:07:40 2022] GET /identity/v3/groups/5dec506f5c864d4fbbe3f31b8461b930 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:41.137536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:41.156904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:41.240890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61b5a0-fb6d-4365-8834-cd5d7cf4ade9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:41.275365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:41.276014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:41.276340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:41.276619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:41.277448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] RBAC: Adding query filter params (group.id=5dec506f5c864d4fbbe3f31b8461b930, role.id=02d9e6449ea143beb203824112640698, scope.project.id=efd554ba71f4463abb4c78dc3cafe52a) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:41.277752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:41.279183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4240888-aa16-4c7c-aeb7-c864cc882e9c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:41.285721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 235/455] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jun 1 04:07:41 2022] GET /identity/v3/role_assignments?group.id=5dec506f5c864d4fbbe3f31b8461b930&scope.project.id=efd554ba71f4463abb4c78dc3cafe52a&role.id=02d9e6449ea143beb203824112640698 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:44.136298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53476089-99ef-4a5f-a95a-dea5467bb1e6 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.136837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53476089-99ef-4a5f-a95a-dea5467bb1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.136837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53476089-99ef-4a5f-a95a-dea5467bb1e6 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.138479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 221/456] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:44.144121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.144511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.144690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.164934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:44.166144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:44.244351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39318-fa3f-48a1-9f08-96627cf327b2 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.245010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39318-fa3f-48a1-9f08-96627cf327b2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.245358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39318-fa3f-48a1-9f08-96627cf327b2 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.247292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 222/457] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:44.254886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 236/458] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:44.256785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1005e-7732-45c6-bfc1-73c7de275619 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.257140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1005e-7732-45c6-bfc1-73c7de275619 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.257627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1005e-7732-45c6-bfc1-73c7de275619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.278767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:44.292499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-50e1005e-7732-45c6-bfc1-73c7de275619 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:44.293287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:44.293749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-50e1005e-7732-45c6-bfc1-73c7de275619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:44.413853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 223/459] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:44.417892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e93dea-b2ba-44ec-8d9a-82272b7ab56c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:44.420213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d976dcc-977d-4408-b33d-82584415893b None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.420576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d976dcc-977d-4408-b33d-82584415893b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.420942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d976dcc-977d-4408-b33d-82584415893b None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.422703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 224/460] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:44.431917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.432421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.432942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.451211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:44.452821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:44.470739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:44.471572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.471857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.472133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.482470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:44.484220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:44.502911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:44.503834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:44.504676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:44.504676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:44.504676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:44.504676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:44.504676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 237/461] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:44.505668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:44.518755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:44.531516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 225/462] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:44.532986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:44.590887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:44.648127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a05c8-ef54-48af-8fd5-d265b26c4a5b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:44.649740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502043c-38b4-4f40-ac4a-80a4c709a018 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:44.685514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:44.686554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.687036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.687574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.688507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:44.689038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:44.690623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:44.697665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:44.699075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.699386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.699669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.705061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 226/463] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:44.706114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:44.713130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:44.714686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-91be717d-d513-4a50-a2c1-e63cdf5aa12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:44.734166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:44.735026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:44.736518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:44.736518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:44.736518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:44.736518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:44.736518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 238/464] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:44.736902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490741e-a89a-45a7-8cbd-18807865b383 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.737334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490741e-a89a-45a7-8cbd-18807865b383 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.737434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490741e-a89a-45a7-8cbd-18807865b383 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.738823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 239/465] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:44.742526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490741e-a89a-45a7-8cbd-18807865b383 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:44.765299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:44.820037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c8f3f-4971-447d-9825-8cf5b3ecb663 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:44.842815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490741e-a89a-45a7-8cbd-18807865b383 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:44.856156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:44.857078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.857476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:44.857990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.858380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.869704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:44.871296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:44.877644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ad6f3-ef77-4e53-958f-f75f8a8e65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:44.878314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ad6f3-ef77-4e53-958f-f75f8a8e65b4 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.878736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ad6f3-ef77-4e53-958f-f75f8a8e65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.879021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ad6f3-ef77-4e53-958f-f75f8a8e65b4 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:44.879762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ad6f3-ef77-4e53-958f-f75f8a8e65b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:44.881470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ad6f3-ef77-4e53-958f-f75f8a8e65b4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:44.890945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:44.892450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:44.893514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:44.893514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:44.893514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:44.893514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:44.895054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 227/466] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:44.896187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:44.997435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 240/467] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:44 2022] POST /identity/v3/users => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:44.998798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:44.999273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:44.999683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.012601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd535a7-f07c-4af4-99cd-4f6e7ddbf027 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:45.017423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:45.024735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:45.027076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:45.057207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:45.058546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.059041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.059541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.060622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:45.060940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:45.062440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:45.081757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 228/468] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:45.082624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:45.122263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 241/469] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:45.123497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:45.222814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b73f94-4d01-4815-af63-5bd034ee8429 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:45.268164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:45.270474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd598b5-637b-408f-99df-4a72ae89270c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:45.270584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:45.270767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.271018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.271317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.272068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:45.272384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:45.273780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:45.289688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 229/470] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:45.290961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:45.311518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:45.312510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.312794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.313156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.314791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:45.316749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:45.400662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:45.449396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 242/471] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:45 2022] POST /identity/v3/users => generated 248 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:45.449999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:45.575377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3efd0-ee51-4003-983f-20f8d2be5d2d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:45.617598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7ecb-c12b-4acb-b289-058f4522252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:45.621119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7ecb-c12b-4acb-b289-058f4522252a None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.621119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7ecb-c12b-4acb-b289-058f4522252a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.621119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7ecb-c12b-4acb-b289-058f4522252a None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.621119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7ecb-c12b-4acb-b289-058f4522252a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:45.622445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7ecb-c12b-4acb-b289-058f4522252a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:45.641690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f596964-e4cd-4a6a-9ca3-be629c63990b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:45.679935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:45.680740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.681040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.681303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.693577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:45.695249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-99386d10-8dc6-4344-93c9-e33c38863a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:45.713358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:45.714222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:45.715094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:45.715094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:45.715094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:45.715094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:45.715094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 230/472] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:45.715844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb59ba-fa34-45a9-8b2a-1ab5f925398d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.716106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb59ba-fa34-45a9-8b2a-1ab5f925398d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.716380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb59ba-fa34-45a9-8b2a-1ab5f925398d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.717766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 231/473] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:45.722272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb59ba-fa34-45a9-8b2a-1ab5f925398d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:45.738663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 243/474] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:45 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:45.740172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.740475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.740756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.763789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:45.765162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:45.774754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:45.825125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:45.848517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 244/475] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:45.872906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:45.882175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb59ba-fa34-45a9-8b2a-1ab5f925398d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:45.958089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:45.958768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:45.959047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:45.959335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:45.960066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:45.960372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:45.961791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:45.984908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 232/476] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:45.992150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:46.046653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f9014-238c-41ac-a4c6-3fa46ecd0808 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:46.084447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:46.085472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:46.085988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:46.086459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:46.087433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:46.089080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:46.120576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8ea81-39d9-4358-bc42-33cdc51828f1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:46.125521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.sql.core [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 01 04:07:46.125521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 01 04:07:46.125521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [parameters: {'id': '1a98c6f8f46a48ad9086c22a359f62e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 01 04:07:46.125521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80710) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result = self._query(query) Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application conn.query(q) Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result.read() Jun 01 04:07:46.146169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return ref.to_dict() Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 01 04:07:46.147091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application next(self.gen) Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application yield resource Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application next(self.gen) Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.session.rollback() Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.force_reraise() Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise self.value Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application session.commit() Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 01 04:07:46.148111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._prepare_impl() Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.session.flush() Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._flush(objects) Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application compat.raise_( Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application flush_context.execute() Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rec.execute(self) Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 01 04:07:46.148977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 01 04:07:46.149842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result = self._query(query) Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application conn.query(q) Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result.read() Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 01 04:07:46.150656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [parameters: {'id': '1a98c6f8f46a48ad9086c22a359f62e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:46.151459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 01 04:07:46.152227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:46.155228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 245/477] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jun 1 04:07:45 2022] POST /identity/v3/roles => generated 144 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:46.155786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:46.169157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:46.184933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:46.185952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:46.186430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:46.186915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:46.187896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:46.189699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:46.196938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:46.274957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 233/478] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:45 2022] POST /identity/v3/users => generated 249 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:46.287325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf4abb-efa3-40f5-a8b1-77e4bb5d16e1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:46.332529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a9704-dddd-4344-995d-44d3965055db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:46.333463 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a9704-dddd-4344-995d-44d3965055db None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:46.333741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a9704-dddd-4344-995d-44d3965055db None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:46.334016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a9704-dddd-4344-995d-44d3965055db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:46.348443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a9704-dddd-4344-995d-44d3965055db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-166a9704-dddd-4344-995d-44d3965055db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:46.358171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:46.359110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:46.360165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 246/479] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:07:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:46.367191 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:46.504649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3524d75-9de5-4451-bae6-91698e2ee0f8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:46.554604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:46.555317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:46.555608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:46.555881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:46.556643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:46.556948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:46.558335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af7c69-0fd8-4e0e-9ab7-406e7ed6a31f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:46.572196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 234/480] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:47.814765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f2bde-d03f-4190-bdb3-54a75ab52bed None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:47.815806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f2bde-d03f-4190-bdb3-54a75ab52bed None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:47.816295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f2bde-d03f-4190-bdb3-54a75ab52bed None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:47.818098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 247/481] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:47.823827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:47.824653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:47.825128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:47.851817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:47.853345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:47.920871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:47.954394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 235/482] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:47.990669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f2bde-d03f-4190-bdb3-54a75ab52bed None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:48.141660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:48.213872 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f2bde-d03f-4190-bdb3-54a75ab52bed None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:48.275130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a915100-7787-4856-a333-4c22cc467e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:48.275961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a915100-7787-4856-a333-4c22cc467e1b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:48.276244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a915100-7787-4856-a333-4c22cc467e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:48.276527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a915100-7787-4856-a333-4c22cc467e1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:48.295500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a915100-7787-4856-a333-4c22cc467e1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-3a915100-7787-4856-a333-4c22cc467e1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:48.303438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:48.304334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:48.305170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 248/483] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:48.310541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:48.400910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b7c8b-82e4-4a55-94a6-8c0860401c91 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:48.401255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b7c8b-82e4-4a55-94a6-8c0860401c91 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:48.401471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b7c8b-82e4-4a55-94a6-8c0860401c91 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:48.403006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 249/484] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:48.420834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:48.421227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:48.421469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:48.459101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a1a8a-057a-48d0-8ded-b4915568e42d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:48.471863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:48.473342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:48.525256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:48.526269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:48.526704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:48.527097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:48.528374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:48.528785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:48.530831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:48.549597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 236/485] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:48.554691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:48.617653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 250/486] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:48.642827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:48.659689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:48.715672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:48.761672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e765521-8845-47ad-8aed-d1e53067095d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:48.769972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ca3f-eaed-4835-8ecb-338f6696a608 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:48.824052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:48.825222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:48.825507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:48.825784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:48.834319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:48.835173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:48.835483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:48.835862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:48.847921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:48.849538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:48.862651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:48.884495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:48.885429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:48.886220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 251/487] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:48.887720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:48.889584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:48.891706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:48.891706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:48.891706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:48.891706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:48.891706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 237/488] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:48.897208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:48.900533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.004232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b57ea-01ea-4b4d-afde-5b4f246fbb7a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.030254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.035696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.043159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17996e2-3e08-4e09-ac49-3de03388888f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.048278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.048953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.049240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.049520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.050277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:49.050586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.052006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8ee7e-cb94-4f86-85ff-6e70bfb4baf8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.067276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 252/489] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:49.067276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80cdab-2511-4651-b71a-4eca33eb2003 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.067276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80cdab-2511-4651-b71a-4eca33eb2003 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.067276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80cdab-2511-4651-b71a-4eca33eb2003 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.067276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 253/490] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:49.068699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80cdab-2511-4651-b71a-4eca33eb2003 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.111859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.112585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.113203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.113311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.114143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:49.114460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.116644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b4619-b281-4ff3-8db8-6b0ca9d0ac41 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.135398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 238/491] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:49.138305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.139692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.140030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.160530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:49.161794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:49.244955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 239/492] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:49.244955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.346213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.362605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97145e5b-93db-4731-977e-d76a5a32d502 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.372468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80cdab-2511-4651-b71a-4eca33eb2003 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.403452 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.404622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.405098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.405577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.413658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.413880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.414672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.415009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.415414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.416274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.427288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.428773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:49.433248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:49.434150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:49.435457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 240/493] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:49.436221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-9f746b3a-7f66-46ce-aae5-4355859352e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:49.450878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:49.451878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 254/494] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc74fe-40f3-4d44-80d5-edb49a8a18dd None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc74fe-40f3-4d44-80d5-edb49a8a18dd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc74fe-40f3-4d44-80d5-edb49a8a18dd None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.458137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 255/495] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:49.458631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc74fe-40f3-4d44-80d5-edb49a8a18dd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.490118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.552003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652c4de-7d9d-48e8-8a93-2e070ebf46a8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.567074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc74fe-40f3-4d44-80d5-edb49a8a18dd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.587920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.588781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.589073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.589356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.601712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:49.602512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.603211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.603502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.603779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.605357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:49.605803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.607251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadca8d1-ed80-41f6-baac-f4ed05441770 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:49.609809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:49.610702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:49.611955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 241/496] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:49.612813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.631616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.650405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 256/497] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:49.651735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.652032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.652315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.670930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:49.672152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:49.726224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e61ccb-9ac4-4ece-b3b5-15c93421dffa None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.740287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 257/498] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:49.741130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.758925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.762008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.762701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.762976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.763256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.764009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:49.764319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.765693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.776987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 242/499] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:49.777781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.840961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.841565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc88e-9278-4af0-a6bb-a69b9bd12755 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.875423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64baea-c29e-4046-8057-ebb78bd997b8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:49.875560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.876153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.876436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.876716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.877466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:49.877772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.879149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b643-522d-4dbc-99f6-1c5abe16a15b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.887424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 258/500] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:49.888635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce95649-99cc-45af-8e25-9647ecc4e7b3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.888921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce95649-99cc-45af-8e25-9647ecc4e7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.889196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce95649-99cc-45af-8e25-9647ecc4e7b3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.890583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 259/501] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:49.892623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce95649-99cc-45af-8e25-9647ecc4e7b3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:49.910157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:49.910808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:49.911097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:49.911405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:49.973587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:49.977017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:49.978482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:49.993038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce95649-99cc-45af-8e25-9647ecc4e7b3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.026002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.026778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.027059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.027353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.037658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:50.041252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:50.045325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:50.046225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:50.047063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 260/502] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:50.048019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.080485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 243/503] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:50.080485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.153126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dbe56-eea6-4058-9cd4-55079d527bda None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.194859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829b7bf-831b-4f65-9f83-bed3c2a605cd None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.196041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.196879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.197193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.197471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.206397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.207897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.211290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-99d253a9-8375-47bc-84b5-e2b506601bfc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:50.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:50.231856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 261/504] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.232807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.233481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.233549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.233549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.233549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.233549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.244813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.246166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:50.246341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.255270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:50.270404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:50.271602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:50.272555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:50.272555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:50.272555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.272555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:50.272555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 244/505] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:50.272555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.293802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.345056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 262/506] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:50.346006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.422393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4038a9-a1a7-435a-9410-df9e2037ae31 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.458006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.458680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.458970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.459254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.460041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:50.460354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.461736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.465975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9aa8e-7a61-42e0-b38e-fee799fa2916 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.469968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 245/507] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:50.470781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.502292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.508371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.510045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.510045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.510045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.520588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.522180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:50.541384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:50.542462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:50.543296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:50.543296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:50.543296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.543296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:50.543296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 263/508] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:50.543592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.571014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.590620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde0056-4656-4e3b-bec7-5363661546cf None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.634638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.635322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.635627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.635905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.636747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:50.637065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.638503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.649711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5282a-e3af-4003-b7b5-560e56877e47 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.677258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 246/509] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:50.678025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.689408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.690044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.690331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.690734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.691455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:50.691682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.693067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.709028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 264/510] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:50.709896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.755450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.785749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0846ea-ed6c-40dc-b672-645833f12f57 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.814957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2e71-e9de-4a3b-a64d-a8f0141fa622 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:50.820013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.820814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.821100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.821378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.827993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:50.831940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.833410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:50.851620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:50.853107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:50.853937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:50.853937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:50.853937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.853937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:50.854152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:50.854152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:50.854567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 247/511] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:50.854637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:50.854637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:50.855191 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.866411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:50.868029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:50.887780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:50.888668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:50.889509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:50.889509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:50.889509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:50.889509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:50.894942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 265/512] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:50.894942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:50.970472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01500d08-42db-42b9-b6e2-df5de223e8a9 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.000636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.008273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.008975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab66fc2-abd1-42f1-b2bb-0f6d2200179b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.009089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.013440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.013440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.013440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:51.013440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.013440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.031164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 248/513] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:51.031930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.045780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.046427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.047067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.047372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.109014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.119067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.120582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.162989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57085af-9d3f-4b28-ba29-0b5d238bf9ab None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.200168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.200971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.201263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.201543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.210407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.211971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.223944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:51.230082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:51.230997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:07:51.248537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:51.249132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 266/514] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:51.249211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 249/515] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:51.249286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.249354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.334808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d9ca0-a430-407e-adb0-4c263734e463 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.338397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d8103-263a-4199-9f1f-8fe0bf13f8c0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.347160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.370821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.371515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.371925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.372198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.372910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:51.373212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.374611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.375255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80961b22-39e0-4435-937c-723411ee419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.376150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80961b22-39e0-4435-937c-723411ee419f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.376451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80961b22-39e0-4435-937c-723411ee419f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.376703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80961b22-39e0-4435-937c-723411ee419f None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.378249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80961b22-39e0-4435-937c-723411ee419f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:51.378249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80961b22-39e0-4435-937c-723411ee419f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.379590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80961b22-39e0-4435-937c-723411ee419f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.390289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 250/516] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:51.391139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.392240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 267/517] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:51.392902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80961b22-39e0-4435-937c-723411ee419f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.448790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.496273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80961b22-39e0-4435-937c-723411ee419f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.509564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9830ce4-5a7f-417b-8e21-54473e4cf765 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.530421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.531214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.531524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.531801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.542867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.544349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.549188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.549984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.550275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.550556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.557372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.558809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:51.562182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:51.563099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:51.564240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:51.564240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:51.564240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:51.564240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:51.564240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 268/518] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:51.564240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:51.577103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:51.578170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 251/519] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:51.579735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.612612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.687381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb1be0-daaf-427c-b5bf-6d5b0ccbede9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.690162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672d8e3-00df-446b-afb0-d8181931466e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.694258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.721404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.722078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.722364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.722639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.723495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:51.723806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.725629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.726097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.726766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.727048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.727348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.728059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.729477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.743100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 269/520] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:51.743953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.852309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 252/521] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:07:51 2022] POST /identity/v3/users => generated 247 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:07:51.852309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.852986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c0ec1-3f1c-4a32-bfef-87fd6d67ea8c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:51.892521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:51.893898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:51.894206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:51.894545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:51.905206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:51.906792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:51.925955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:51.926942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:51.927994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 270/522] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:51.929110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:51.955911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.965621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:51.972538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878e250-d9b6-42da-9b8c-a6e21a5f8432 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.014991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.015704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.015993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.016270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.017885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:52.018195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.019634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.031803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 253/523] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:52.032712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.062583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6338551-c685-496b-9734-8ba423d2fe01 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.107291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.107953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.108244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.108526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.109590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:52.109909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.111671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.133824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 271/524] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:52.134696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.156440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7c202-5105-4dcc-bfb1-29c3aca51fe0 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.193589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.194643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.195225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.195548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.212877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:52.223818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:52.239734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c174b6-4fa6-4273-8b1a-07151e06e9b2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.271575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.273038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.275213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.275921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.276208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.276491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.277337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:52.279277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.279277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.296736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 272/525] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:52.297582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.390630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 254/526] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:52.396105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.410603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997cd6b-3ffb-49e3-b198-79723677a43e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.447643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:52.450991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.451831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.452113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.452387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.459326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.460781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:52.475480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:52.476379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:52.477230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 273/527] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:52.477698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.480483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:52.509849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b6352-8fed-4045-82a9-8f6a61cbab24 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.544579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396064-056d-4266-b342-5f8059765edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.545494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396064-056d-4266-b342-5f8059765edc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.545964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396064-056d-4266-b342-5f8059765edc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.546449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396064-056d-4266-b342-5f8059765edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.547543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce396064-056d-4266-b342-5f8059765edc None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=56036136066d412dac682273ebf65ab9) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:52.548548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce396064-056d-4266-b342-5f8059765edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.550342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce396064-056d-4266-b342-5f8059765edc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.556051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 255/528] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/role_assignments?user.id=56036136066d412dac682273ebf65ab9&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:52.556833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396064-056d-4266-b342-5f8059765edc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.600705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b05af-b6e8-45f5-9d6d-2e3103a69dd4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.648564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.650266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.650758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.651213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.692240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396064-056d-4266-b342-5f8059765edc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.706806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:52.744444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.745373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.745670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.745953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.752901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.754431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.756960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:52.760731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.762223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:52.784014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:52.785170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:52.786126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:52.786126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:52.786126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:52.786126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:52.786126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 256/529] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:52.786861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.895273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 274/530] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:52.896826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:52.925339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9e9b-1894-4723-bb77-b08a9c903d61 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:52.975220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:52.976384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:52.976758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:52.977003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:52.977846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:52.978162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:52.980282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.014421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 257/531] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:53.014421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.031456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea973f-cfc6-4f33-b190-cb00ad96e461 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.034164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:53.053897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:53.074189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.075648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.076127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.076572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.077516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:53.077996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.080511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.093278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 275/532] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:53.094674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.165406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e69fd-0b33-4d40-a91e-96b033c66eb5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.219096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.222812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.222812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.222812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.227557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8af814-54fd-461c-95b3-fcb48e05c3da None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.236893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.238898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:53.263674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:53.264758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:53.266111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:53.266111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:53.266111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:53.266111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:53.268431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 258/533] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:53.269508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.285109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.285803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.286497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.286794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.287996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=2e0959756bd54aa58afb14b9b45d29ad) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:53.288323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.291372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.302820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 276/534] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/role_assignments?user.id=2e0959756bd54aa58afb14b9b45d29ad&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:53.311575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.349889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:53.452532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:53.465454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29acea47-3e63-406c-bf9b-7c1b8ffbdda6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.496467 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036bb6-6466-41f1-9b35-7f656f644618 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.535063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.535905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.536247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.536477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.537442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.538121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.538412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.538698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.539475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:53.539803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.541190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.543980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.545440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.554756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 277/535] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:53.560084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:53.561366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:53.562171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:53.563003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 259/536] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:53.568151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.677902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa23c-f39a-4501-9606-448c84968610 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.725825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.726503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.726794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.727077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.727923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:53.728237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.729695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.735608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:53.761653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 260/537] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:53.762992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.772723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:53.809343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffadf8-e3e4-460e-84ca-ebc4ff2b0794 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.873232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.874594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.875078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.875572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.883371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917635b9-e32f-4853-bd03-58b08084d88c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:53.898920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.900499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:53.927850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:53.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:53.929641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 278/538] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:07:53.931441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:53.932611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:53.933072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:53.933494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:53.934539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=3f4e0e17040d41bf91cf4022dc4b4d4e) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:53.934808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:53.935114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:53.936864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:53.950382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 261/539] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/role_assignments?user.id=3f4e0e17040d41bf91cf4022dc4b4d4e&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:54.049189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a60910-2d33-4587-89df-900623e21df2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:54.087063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:54.087766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:54.088053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:54.088333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:54.089133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:54.089446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:54.090909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3600bd-69ee-4fb9-8ded-2d840e55ea57 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:54.104516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 279/540] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:54.110538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:54.177781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:54.242707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373906-d6b2-42b7-9b2e-488448c61991 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:54.267125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48898c7-ce57-47dd-a204-dc77441f2aad None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:54.267361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48898c7-ce57-47dd-a204-dc77441f2aad None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:54.267653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48898c7-ce57-47dd-a204-dc77441f2aad None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:54.269294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 280/541] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:54.281655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:54.282216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:54.282328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:54.295677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:54.305522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:54.306131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:54.306417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:54.306697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:54.307573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=885cc3a3c23c451c9ad7526e7fa83639) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:54.307889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:54.309331 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:54.312116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:54.313318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:54.320612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 262/542] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:54 2022] GET /identity/v3/role_assignments?user.id=885cc3a3c23c451c9ad7526e7fa83639&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:54.429852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 281/543] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:54.472294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:54.603360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f1cc-13fa-42de-a136-dbf8b1258af3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:54.637424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:54.652985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:54.654048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:54.654144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:54.654441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:54.669174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:54.677770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:54.678626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:54.679471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:54.683636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 263/544] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:54.685395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:54.837207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305f10d-859f-4ff4-85a7-b6811c40d7bf None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:54.880282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:54.881140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:54.881420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:54.881693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:54.882509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:54.882815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:54.884339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:54.895641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:54.899040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 282/545] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:54.904133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.044090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9c96a-fe25-4e2e-8804-2bfc3bb2a49c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:55.087592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:55.088398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.088682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.088961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.099948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:55.101603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:55.119330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:55.120200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:55.121201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:55.121201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:55.121201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:55.121201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:55.121201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 264/546] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:55.128477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.252118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c15db-0500-4350-b44f-a1da856c7fe9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:55.297293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:55.298577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.298760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.299038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.299860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:55.300167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:55.301767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07544338-5c17-4af8-8611-10b068e71f79 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:55.314712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 283/547] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:55.319804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.359703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:55.426739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb6115-e525-4152-b5e6-9b8483f90344 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:55.464648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:55.465473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.465816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.466102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.474341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:55.475910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-29f06291-366d-484d-81cd-ec415980fa16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:55.491918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:55.493255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:07:55.494205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:55.494734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff755996-aa73-4e8f-b32a-b5462b01e21f None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.494734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff755996-aa73-4e8f-b32a-b5462b01e21f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.494734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff755996-aa73-4e8f-b32a-b5462b01e21f None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.496081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 284/548] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:55.496958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 265/549] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:07:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:55.502631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff755996-aa73-4e8f-b32a-b5462b01e21f None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.506571 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.506863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.507146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.524239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:55.525532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:55.595987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 266/550] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:07:55.636979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.655682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:55.691002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:55.733561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff755996-aa73-4e8f-b32a-b5462b01e21f None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:55.790019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:55.790705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.790990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.791584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.792430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:55.792742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:55.794317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:55.813206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 285/551] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jun 1 04:07:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:55.818979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.855715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5c2e9-95fe-4ca1-9066-c7ea950593e5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:55.924031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:55.924849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:55.925140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:55.925424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:55.944970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:55.952951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:55.953889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:55.954949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 267/552] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:55.959766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:55.986855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61caff4-119b-4920-9135-f065f3e92e6a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.019671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.057061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6e70e-97b0-418c-b5e8-4b0b696b8682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.058553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e70e-97b0-418c-b5e8-4b0b696b8682 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.059602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e70e-97b0-418c-b5e8-4b0b696b8682 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.059901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6e70e-97b0-418c-b5e8-4b0b696b8682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.105933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.121287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567097c-f812-49d6-9d51-2544baaf8257 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.156854 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.157709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.158069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.158454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.159274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:56.159684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.161138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfb407-d0cc-463d-b39f-527fd872482c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:56.163210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6e70e-97b0-418c-b5e8-4b0b696b8682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.164935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6e70e-97b0-418c-b5e8-4b0b696b8682 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:56.170234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 268/553] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:07:56.171674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da55118-ee85-4b44-a30e-f87f1c78254a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.172065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da55118-ee85-4b44-a30e-f87f1c78254a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.172446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da55118-ee85-4b44-a30e-f87f1c78254a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.178395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 269/554] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:56.178395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da55118-ee85-4b44-a30e-f87f1c78254a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.286605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da55118-ee85-4b44-a30e-f87f1c78254a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.297012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 286/555] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:07:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:07:56.299334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.299689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.300035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.317913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:56.319236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:56.323646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.324449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.324790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.325070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.331658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.335983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.337535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:56.345471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:56.357071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:56.358053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:56.359312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:56.359312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:56.359312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:56.359312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:56.359312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 270/556] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:56.364906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.392465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 287/557] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:56.393346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.470199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf19768-7998-497c-8162-37270c8f6713 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.507648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.508560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.508917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.509147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60246b-f666-4a4d-a2da-5bf2498394ce None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.509316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.524171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.526543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:56.550646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:56.553202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:56.555565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:56.555565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:56.555565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:56.555565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:56.555565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 271/558] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:56.555565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.562616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.563387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.563677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.563956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.565269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:56.565584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.567477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:56.587137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 288/559] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:56.588017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.598188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.611473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.695436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a2bf9-7219-4b29-9896-369de81d7965 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.701997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b39a1-f463-467f-8fe9-e7ba557ac6a7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.740015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.740683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.740971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.741253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.742019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:56.742287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.743705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:56.761494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.762249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 289/560] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:56.763157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.763727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.764437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.765146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.783686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:56.792348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:56.793558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:56.794608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:56.796943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 272/561] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:56.797955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:56.855149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.912017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda043c-5b50-4993-b304-52374f719132 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.958727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25060a5-828a-4ada-8823-fa67b3cc0f8b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:56.968933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:56.970763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:56.971232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:56.971745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:56.978166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:56.982653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:56.985109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b5be8262-c7ab-4295-b97d-b34e05188ef6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:57.001687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:57.003243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 290/562] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56402bf3-d2bb-4ba5-b0f2-10218dc55b85 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.004760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56402bf3-d2bb-4ba5-b0f2-10218dc55b85 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.005729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56402bf3-d2bb-4ba5-b0f2-10218dc55b85 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.006282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 291/563] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:57.011472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56402bf3-d2bb-4ba5-b0f2-10218dc55b85 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.018491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.019456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.019732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.020024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.027849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.029332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:57.053987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:57.054976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 273/564] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:57.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.120941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56402bf3-d2bb-4ba5-b0f2-10218dc55b85 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.152353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.161455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.162126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.162493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.162914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.164273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:57.164943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.165046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a7382-7554-4e23-9119-c9dd22ba0c06 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.167747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd7446-734d-4567-94c8-2ec340cc5a39 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.180013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 292/565] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:57.181303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7415-d950-421e-9863-063d6669535e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.181595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7415-d950-421e-9863-063d6669535e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.181887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae7415-d950-421e-9863-063d6669535e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.199524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-43ae7415-d950-421e-9863-063d6669535e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:57.200729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-43ae7415-d950-421e-9863-063d6669535e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:57.206236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.206907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.207194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.207503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.208347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:57.208659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.210118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.227342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 274/566] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:57.232236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.261206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.268256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 293/567] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:57.269592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12018845-23df-4d62-af7b-8459d5a39a2a None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.269885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12018845-23df-4d62-af7b-8459d5a39a2a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.270168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12018845-23df-4d62-af7b-8459d5a39a2a None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.271440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 294/568] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:07:57.272134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12018845-23df-4d62-af7b-8459d5a39a2a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.352296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2ec81-cbfd-4223-9e39-d9847f4c1a95 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.380259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12018845-23df-4d62-af7b-8459d5a39a2a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.387471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.388141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.388435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.388720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.389579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:57.389900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.391471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.400457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.404456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 275/569] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:57.405291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.419126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.419956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.420244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.420527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.431589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.433054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-a08d1fce-c913-41c2-88ea-41b2184a4386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:57.451165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:57.452349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:57.453652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:57.453652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:57.453652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:57.453652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:57.453652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 295/570] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:57.453652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.454274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.454274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.470404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:07:57.471599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:07:57.496955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.509454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fb883-b5e9-4315-b934-31ec5923ad6d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.539396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 296/571] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:07:57.540295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.558750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.559588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.566100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.566100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.579763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.581229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:57.598842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:57.599997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:57.600944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:57.600944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:57.600944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:57.600944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:57.600944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 276/572] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:57.601234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.644695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8725903-fd81-428e-9b0b-13419ceac4e5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.678159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.678967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.679254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.679560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.690473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:57.700639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:57.703890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.704745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 297/573] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:57.705562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890c1ea-169f-4225-a03f-cd3b00a7b929 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.705645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.738521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.739182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.739507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.739792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.740635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=400527de384349f58866dd9e39468307) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:57.740955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.742412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.747958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 277/574] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/role_assignments?user.id=400527de384349f58866dd9e39468307&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:57.748763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.762301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.819138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e164e-8c30-4ac0-83e9-46814014d850 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.875032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.875829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.876151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.876383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.877137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:57.877451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:57.878906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:57.884104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f822fe-f0a1-4d53-ad77-13635d544ff0 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:57.893402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 298/575] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:57.894584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:57.919597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:57.931740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:57.932574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:57.932909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:57.933186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:57.946309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:07:57.954374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:57.955387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 278/576] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:57.956630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.015796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016cf9-0034-40d4-a25a-f1e22fae7eb4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.028351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.055630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.056343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.056579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.057185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.057587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:58.057895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.059292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.070617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5a0ca-410f-44f6-9cb2-a04356823954 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.072467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 299/577] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:58.073228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.116366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.117553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.117841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.118119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.119221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:58.119712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.121411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.131421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 279/578] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:58.132124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.166160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.187576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3cf3f-89bd-4ff5-a130-20f2f1162ab0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.232815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.234682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.235185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.235691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.244968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.246415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.247504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161faa7-ff9d-4697-9307-740a22fa4dd7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:58.261414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:58.262425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 300/579] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:58.263428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.283693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.293811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.294488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.294805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.295038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.295829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:58.296138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.297517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.315192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 280/580] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:58.316519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.386965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7853c-8b45-4c22-82f4-c616fe03cbfd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.422906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.423842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.424064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.424289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.431315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.432129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778d489-6a49-496f-acaf-e3a5c86aac51 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.432803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.435602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:58.448549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:58.449650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 301/581] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:58.450789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.481350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.482580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.483009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.483757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.496478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.497938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:58.517845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:58.518771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:58.519702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:58.519702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:58.519702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:58.519702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:58.519702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 281/582] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:58.519981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.569052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0b7f8-343d-4296-bcab-8e74f7cecfd5 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.574030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.606698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.607395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.607679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.607953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.608790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:58.609097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.610594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.634871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 302/583] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:58.634871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.648174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23c5e-cd2b-412c-96f0-29499f7dccc3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.689411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.690226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.690515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.690795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.703032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.705562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.707042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:58.725150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:58.726214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:58.727117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:58.727117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:58.727117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:58.727117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:58.727117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 282/584] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:58.727934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.756892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75aac6-bd48-4349-adf1-8694974508e8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.791520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.792261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.792550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.792821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.793603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:58.793913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.795383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.808608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 303/585] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:58.809448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.836136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.847865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d673ef-e7b4-4404-a3f8-4376fc47ea6f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.896600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.897265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.897552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.897821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:58.898549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:58.898854 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:58.900260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:58.913259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 283/586] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:58.913893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:58.942964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73df3-7c3e-4cca-9b90-6d33af99cad0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:58.964407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:58.987958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:58.988755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:58.989044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:58.989324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.003349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.006189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:59.025403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:59.026866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:59.028340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:59.028340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:59.028340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:59.028340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:59.028340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 304/587] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:59.028340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.033470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf884b-5ec1-4296-83f8-71f527abda34 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.083611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.084306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.084588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.084862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.085588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:59.085896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.087269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.103695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 284/588] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:59.103695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.119244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:59.158158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f48cfb-19d8-41d7-9329-3143ef99a98c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.207515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.208411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.208691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.209205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/56036136066d412dac682273ebf65ab9/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.224368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=56036136066d412dac682273ebf65ab9, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.225741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.228904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ed62a-9536-477e-9c62-bf00742c23b0 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.263032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.263889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.264161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.264432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.271606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.276950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:59.288375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:59.289553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 285/589] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:59.290779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.315785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 305/590] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:07:59 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/56036136066d412dac682273ebf65ab9/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:07:59.315785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.344260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:59.396484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:59.449011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969bf6-f74f-4680-95d0-be970d1f497c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.456128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a80bf5-e494-4c4a-89db-49986b463149 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.489583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.490224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.490506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.490780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.491555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:59.491866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.493361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.507632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.508215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 286/591] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:59.508454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.508740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.509018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.509134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.519505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.521519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:07:59.542685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:59.543795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 306/592] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:59.546263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.651447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2378db6-b930-4de8-b0ec-92ea09457d44 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.675715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:59.682173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089ab8f-e962-4a35-b9da-9bb564a8138b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:59.696310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.697021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.697313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.697593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.698470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:59.698791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.700338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.724572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 287/593] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:59.731605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.739438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.740303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.740526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.741042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.749828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.751484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:07:59.767274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:59.768239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 307/594] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:59.769716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.873234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beefd4-7c57-4c44-9645-0f6334486a79 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.882300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0185-e0c9-4c9c-ac8e-da78f24a00f4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:07:59.910406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.911342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.912061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.912254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.913238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:07:59.913624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.915325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.920686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:07:59.921676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:07:59.922102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:07:59.922562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:07:59.932124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:07:59.934707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:07:59.936240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:07:59.936761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 288/595] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:07:59.937927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:07:59.955192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:07:59.956155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:07:59.957058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:07:59.957058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:07:59.957058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:07:59.957058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:07:59.960864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 308/596] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:07:59.961713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:07:59.994999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.049621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf87121-3116-4518-9c17-3572ea8158d5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.064416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78086d26-a925-4f24-a944-e7cc560aa2ec None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.086923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.087627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.087916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.088195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.088994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:00.089309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.090734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.101123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.101918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.102371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.102620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.103347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 289/597] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:00.103935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.118319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.120363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-93295101-3e4e-436f-aec6-a58552b9489a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:00.139553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:00.140685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:00.142456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:00.142456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:00.142456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:00.142456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:00.144011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 309/598] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:00.145441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73394-d9b4-42cd-8d67-8d04afc4a031 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.145888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73394-d9b4-42cd-8d67-8d04afc4a031 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.146303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b73394-d9b4-42cd-8d67-8d04afc4a031 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.148021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 310/599] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:00.148940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73394-d9b4-42cd-8d67-8d04afc4a031 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.200712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.218762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ad6d5-ad70-4dce-af07-69dc1fd6c0e3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.229463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.256112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.256700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.256982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.257257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.257973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:00.258083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b73394-d9b4-42cd-8d67-8d04afc4a031 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.258276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.259779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.272900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 290/600] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:00.278296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.294620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc54160-693e-4e7c-9067-7d4d9f90087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.295543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc54160-693e-4e7c-9067-7d4d9f90087d None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.295829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc54160-693e-4e7c-9067-7d4d9f90087d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.296115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc54160-693e-4e7c-9067-7d4d9f90087d None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/2e0959756bd54aa58afb14b9b45d29ad/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.311119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc54160-693e-4e7c-9067-7d4d9f90087d None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=2e0959756bd54aa58afb14b9b45d29ad, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.312623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc54160-693e-4e7c-9067-7d4d9f90087d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.378858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dac2f5-9d1f-46e0-b03d-9d9810f19724 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.379765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 311/601] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:00 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/2e0959756bd54aa58afb14b9b45d29ad/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:00.381268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.381609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.381897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.401736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:00.403223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:00.430105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.431273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.431755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.432179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.433069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:00.433525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.435080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.450030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 291/602] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:00.451242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.487541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.496633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 312/603] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:00.498034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.527990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.594960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d180a-9e24-4387-acd1-0fe31e61287e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.615686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa95cc8-c058-4010-9dc9-64bdbe6c75eb None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.643047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.643919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.644212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.644497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.652066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.653545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.662732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.663567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.663850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.664305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:00.667960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:00.668842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:00.669715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 292/604] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:00.670668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.679321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.680846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:00.699384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:00.700320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:00.701240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 313/605] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:00.701715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.795431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.811962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60519ea9-cb26-4e01-8e28-3633dfbfa2c7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.812527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:00.821522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c875ea-e79b-4630-8d96-a76726823b50 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:00.849930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.850923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.851396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.851778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.858831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:00.859553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:00.859838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:00.860177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:00.861464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:00.861773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:00.863346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:00.865819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:00.873982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:00.874961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:00.876080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 293/606] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:00.877675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.881849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 314/607] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:00.883219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:00.988429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746522c1-650f-4c38-8bc3-9ef921f94ccc None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.011503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11171b7d-058b-479f-bf1c-a0c80e1ce6f1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.025014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3123618-cd60-437d-a750-83277030e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.025691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3123618-cd60-437d-a750-83277030e990 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.025975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3123618-cd60-437d-a750-83277030e990 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.026243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3123618-cd60-437d-a750-83277030e990 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.027128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3123618-cd60-437d-a750-83277030e990 None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:01.027472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3123618-cd60-437d-a750-83277030e990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.028868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3123618-cd60-437d-a750-83277030e990 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.042916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 315/608] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:01.043838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3123618-cd60-437d-a750-83277030e990 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.048821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.049423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.049706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.049981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.050778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:01.051089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.052568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.066582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 294/609] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:01.067481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.079103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:01.079770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:01.162268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3123618-cd60-437d-a750-83277030e990 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.177158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35026604-2100-4a62-83d2-a5222526a7b5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.199792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.200786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.201213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.201597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/3f4e0e17040d41bf91cf4022dc4b4d4e/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.213255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.214062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.214351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.214627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.218956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=3f4e0e17040d41bf91cf4022dc4b4d4e, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.221389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.226286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.227763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:01.245764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:01.268074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:01.268973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:01.268973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:01.268973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:01.268973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:01.268973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 295/610] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:01.268973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.313160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:01.364144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 316/611] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:01 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/3f4e0e17040d41bf91cf4022dc4b4d4e/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:01.366356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.369814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a4896-b959-414e-a8c3-891c90c66cad None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.441017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.443175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.445020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.445480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/885cc3a3c23c451c9ad7526e7fa83639/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.478179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=885cc3a3c23c451c9ad7526e7fa83639, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.480781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.499652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:01.557463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 296/612] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:01 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/885cc3a3c23c451c9ad7526e7fa83639/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:01.557660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc83d0-fdfb-4943-b2be-d9d3e83f85ae None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.599763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.600359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.600640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.601182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.602680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:01.603232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.604780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbfcf2-2b26-42a3-9beb-cc66f8143d48 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.627359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 317/613] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:01.633716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.677488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d86ec-1495-4494-bb5b-78db9682b26f None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.677703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d86ec-1495-4494-bb5b-78db9682b26f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.677923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d86ec-1495-4494-bb5b-78db9682b26f None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.680204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 318/614] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:01.687265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.688970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.689355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.714121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:01.715701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:01.762258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a970bd-c84a-42af-ab50-51477dfb3621 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:01.775407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:01.816517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:01.819172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:01.819812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:01.820290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:01.835533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 319/615] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:01.836709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:01.838405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:01.861000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:01.861986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:01.862920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:01.865200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 297/616] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:01.870478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.870637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:01.932868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:01.998984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bf313-a3b8-4ae1-bfba-b25c2af6f0c3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.014024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199c52-349a-40f3-9463-3d3d2ad9a507 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.042326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.044221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.046056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.046491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.049333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:02.049801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:02.051447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:02.087949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.089338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.091409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 298/617] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jun 1 04:08:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:02.091862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.092293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.099025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.117290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:02.131712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:02.133514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:02.134627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:02.135670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:02.140732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 320/618] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:02.148768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.245998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85489f22-b652-4ff1-9a3b-8e338f6bad20 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.286553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.289851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.290296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.290821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.306353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:02.308103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:02.336948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af2ce-6782-4443-adf8-b99842c8d56b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.347710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:02.348849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:02.350261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:02.351392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:02.351392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:02.351392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:02.351392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:02.353447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 299/619] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:02.359295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.410112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.410993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.411450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.411735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.412492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:02.412803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:02.414187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:02.437005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 321/620] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:02.440027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.525354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:02.564907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918f8a2-33e4-482f-843f-ac8eda71f8b4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.612253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.612934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.613222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.613503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.614232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:02.614542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:02.615961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:02.637282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 300/621] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:02.642783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.649300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3da93-6319-44f5-b671-db186d49cac4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.713668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.714386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.714668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.714941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.733765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:02.744703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:02.746677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c46a89-7637-4c1c-b484-0362f1f7a6da None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:02.770412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:02.771568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:02.772603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:02.772603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:02.772603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:02.772603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:02.772603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 322/622] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:02.780176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.785890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:02.786901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:02.787455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:02.787913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:02.795544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:02.797264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:02.815661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:02.819149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:02.820044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:02.820917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 301/623] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:02.827437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:02.830504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:02.989362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03445c19-6f4c-4b3b-a9bc-c323aca0bc2a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.014061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c374ea-356d-4178-a67d-cbcf5099834e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.033269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.034089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.034500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.034790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.035787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:03.036109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.037907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6188-13c7-4c70-8452-0c832920c7f7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.054935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 323/624] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:03.056316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01303510-2c6f-478d-ab48-ef23108beedf None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.056613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01303510-2c6f-478d-ab48-ef23108beedf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.056895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01303510-2c6f-478d-ab48-ef23108beedf None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.058377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 324/625] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:03.059787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01303510-2c6f-478d-ab48-ef23108beedf None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.065412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.066090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.066372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.066646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.067473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:03.067785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.069249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ec190-6585-4739-99b6-6910d7212805 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.086147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 302/626] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:03.087754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.088196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.088485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.111275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:03.112952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:03.163754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:03.182198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:03.185105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01303510-2c6f-478d-ab48-ef23108beedf None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.200000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 303/627] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:03.200842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.226035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.227029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.227488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.227871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.238452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.241828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:03.261590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:03.269198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 325/628] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:03.270154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.326200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5858cd-495e-43f5-9050-7f289b84ee34 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.368654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.369707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.369997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.370280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/400527de384349f58866dd9e39468307/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.388242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f549f-274b-4389-a6be-bd9eadb2d119 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.390201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=400527de384349f58866dd9e39468307, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.392498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.423822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.424739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.425089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.425430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.434044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:03.437133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:03.445019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:03.445936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:03.446792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:03.450915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 326/629] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:03.451653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.465456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 304/630] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:03 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/400527de384349f58866dd9e39468307/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:03.470168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.526016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:03.610321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae43f08-3134-4a3c-a436-df6e7ac6eef4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.655333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.656219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.656507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.657461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.658244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:03.658556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.659991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.670827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 305/631] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:03.680636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.694175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b076fe-79a9-47ab-ac77-289459a5d091 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.744024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.745088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.745382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.745976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.746854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:03.747394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.749259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.777208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 327/632] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:03.787255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.828397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ee046-dcf6-4ef9-b9e0-1cf28908fcf1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:03.831637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:03.871950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:03.872936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:03.873222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:03.873499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:03.886484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:03.888040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:03.900426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:03.910025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:03.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:03.919738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:03.919738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:03.919738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:03.919738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:03.919738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 306/633] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:03.919738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:03.981024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cb03f-5f98-4e00-9ccb-83500aa52071 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.024917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fc693-11a0-4b54-b5e5-f62c3e0f2a4e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.061169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.062080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.062516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.062935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.064372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:04.064820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:04.066343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:04.072020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa718e-ec79-4193-8178-c8945cfa64c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.072690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa718e-ec79-4193-8178-c8945cfa64c6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.072983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa718e-ec79-4193-8178-c8945cfa64c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.073266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa718e-ec79-4193-8178-c8945cfa64c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.080155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 307/634] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:04.085703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.115733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.194898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caca84-a947-4b8f-a92f-65976ed7d5dc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.212915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa718e-ec79-4193-8178-c8945cfa64c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:04.215529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa718e-ec79-4193-8178-c8945cfa64c6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:04.238583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.239413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.239701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.239973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.247213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:04.248661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-3d05a297-f6b5-4ea8-9056-c40660a2673a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:04.264648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:04.271068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 308/635] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33147c8-7cc4-4f3a-a69d-7944d5cb418b None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.272585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33147c8-7cc4-4f3a-a69d-7944d5cb418b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.273294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33147c8-7cc4-4f3a-a69d-7944d5cb418b None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.273294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 309/636] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:04.274640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266406e-e7d4-40a9-9804-dd628c08a057 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.274926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266406e-e7d4-40a9-9804-dd628c08a057 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.275208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266406e-e7d4-40a9-9804-dd628c08a057 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.279356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 310/637] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:04.279356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266406e-e7d4-40a9-9804-dd628c08a057 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.282778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.336520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 328/638] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:08:04.337736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c902341-cef0-4566-a689-5aaf0c2d5f99 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.338028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c902341-cef0-4566-a689-5aaf0c2d5f99 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.338312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c902341-cef0-4566-a689-5aaf0c2d5f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.354875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0c902341-cef0-4566-a689-5aaf0c2d5f99 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:04.356302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0c902341-cef0-4566-a689-5aaf0c2d5f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:04.360248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.385877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266406e-e7d4-40a9-9804-dd628c08a057 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.423418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.424095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.424378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.424655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.425483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:04.425789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:04.427222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:04.435438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 329/639] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:04.435438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.435438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.435438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.444999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 311/640] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:04.445892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.460903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:04.462080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:04.532915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 330/641] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:04.533777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.552543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.563741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7a8b-03a2-4cdf-b3fc-acaf05d7ce2b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.594939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.600278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.601393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.601506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.601702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.611584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:04.613037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:04.632021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:04.632894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:04.633840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:04.633840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:04.633840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:04.633840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:04.633840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 312/642] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:04.634123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.648228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996f790-36a2-423e-b679-c88bc7e661b6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.683245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.683903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.684192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.684465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.738875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d73d-1f0d-4ebf-8edf-1c95ccf2fc64 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.754566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:04.756303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:04.777294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.778101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.778386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.778662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.778934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.789262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:04.797225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:04.798326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:04.799210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 313/643] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:04.799961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.847611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:04.874857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 331/644] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:08:04.876324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.906670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ddc08-8edf-4ac3-ab5d-ad6105e8d856 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:04.944828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:04.945627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:04.945932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:04.946206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:04.956564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:04.964398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:04.965343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:04.966244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 314/645] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:04.967043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:04.987926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf825b-38b8-4999-9a2d-88d93c6e4927 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.006376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.024395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.025053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.025337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.025614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.026327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.026623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.028036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.040517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 332/646] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:05.041327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.078019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f073d3-8897-4f8f-8633-459660748f0c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.082008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.118888 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.119514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.119803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.120084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.120831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.121141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.122536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.133444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 315/647] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:05.134527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.148585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42a748-0a4b-43e9-9277-7106e71cbc21 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.186157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.187246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.187639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.187914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.201903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.203395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:05.221004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:05.222327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:05.223200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:05.223200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:05.223200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:05.223200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:05.223200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 333/648] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:05.223200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.253239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5da72d-9026-4f33-8b17-a3a8e7501d59 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.261326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.289003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.289672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.289952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.290225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.290972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.291287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.292862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.306697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 316/649] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:05.306697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.332748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.338549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981799d7-82e0-49ea-b138-df57254aad19 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.373955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.374663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.374944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.375219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.382309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.383792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:05.397860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:05.398768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:05.399722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 334/650] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:05.400247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.410710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494d0cd-d2e1-4c4f-8c8b-557d2852ca31 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.446653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.447731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.448018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.448291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.458774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.460247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:05.484602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:05.486241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:05.491813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:05.491813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:05.491813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:05.491813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:05.491813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 317/651] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:05.491813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.522303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.532202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea618b0-4dc0-42b2-a4e8-da7858475d9d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.566505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.567165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.567476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.567752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.568471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.568775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.570148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.580643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.581639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 335/652] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:05.582473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.607032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fa398-4524-4bf7-af40-4c35f7f5ef28 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.649583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.650623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.651072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.651533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.662832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.664603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:05.683493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:05.684665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:05.685676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:05.685676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:05.685676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:05.685676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:05.687513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 318/653] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:05.688474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.702591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6364cde3-0d70-4d2e-ae56-1aa2453477c6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.739323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.739999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.740270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.740547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.741322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.741632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.743219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.756208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 336/654] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:05.761675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.774495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.797582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139f42f-d138-4cb0-b9db-02a5b1cc8c6f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.826617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:05.832440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.833107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.833388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.833662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.834382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.840383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.840383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.848236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 319/655] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:05.849037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.874400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403c12-766f-4136-b757-a2f0246c8ffd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.909840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.910602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.910892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.911170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.918330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.919834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:05.933460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:05.934586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 337/656] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:05.935680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:05.957482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3eca1-ddb1-4c0e-98a4-6a2350801f3e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:05.990402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:05.991054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:05.991367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:05.991646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:05.992360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:05.992792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:05.994161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.004556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.005073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 320/657] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:06.005996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.036309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fea8e-78a9-4238-a926-0c54e097522b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.060579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.069555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.070205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.070494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.070774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.071637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=56036136066d412dac682273ebf65ab9) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:06.071955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.073409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b287ab-1308-481e-ac97-63d65b732f74 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.079896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 338/658] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:05 2022] GET /identity/v3/role_assignments?user.id=56036136066d412dac682273ebf65ab9&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:06.081148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4da80-921c-412d-8e23-8ae7c818f6da None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.081438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4da80-921c-412d-8e23-8ae7c818f6da None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.081722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4da80-921c-412d-8e23-8ae7c818f6da None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.083102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 339/659] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:06.088122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4da80-921c-412d-8e23-8ae7c818f6da None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.113613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bde2c8-3585-4b6c-a348-f3d603079b16 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.154955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.155797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.156085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.156364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.165652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.167072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:06.183597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:06.184689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:06.185831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 321/660] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:06.186449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.211022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4da80-921c-412d-8e23-8ae7c818f6da None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.252273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.253110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.253344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.253590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.254390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:06.254700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.256452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7836b-95cb-4ede-803e-4e7aea146135 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.258523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.273646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 340/661] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:06.273646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.273646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.273646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.292402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:06.293796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:06.315338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a26345-bcec-45cd-bed1-e58d10aae827 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.331334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.358678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.359556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.359876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.360160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.369284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.370769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.374046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 341/662] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:06.375039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:06.386081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:06.387050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 322/663] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:06.388161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.489241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0bc72-c72a-4a09-acbb-7db7e0a16fe1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.493535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d49c74-f03f-44d0-90a5-ece2597f2afd None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.514276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.525241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.526075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.526950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.526950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.527653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:06.527966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.529420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.533133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.533782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.534065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.534339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.535172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=2e0959756bd54aa58afb14b9b45d29ad) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:06.535525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.536931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.543021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 323/664] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/role_assignments?user.id=2e0959756bd54aa58afb14b9b45d29ad&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:06.543935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.567355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 342/665] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:06.567355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.591000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.677421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126bf4-4917-45fb-a8b7-41a30560b315 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.696606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9199344-33c0-48ed-9047-9184a7d4f51c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.718957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.719984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.720418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.720844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.722078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:06.722566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:06.724333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:06.734189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.735556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.735839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.736117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.747045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 324/666] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:06.748138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.748767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:06.761340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:06.762530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:06.763613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 343/667] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:06.764371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:06.817790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.871350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:06.879678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9607899-95fa-4099-b0c7-b87dc5017c1d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.895848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00607d3e-25ba-487a-a94e-16d489f5ef0a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:06.932024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.932683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.932970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.933249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:06.944857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:06.945912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:06.946402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:06.946905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.014003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.015433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.026688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.028573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.075345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.133469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.140035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 344/668] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:08:07.141872 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.148160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 325/669] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:08:07.149773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.275998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df00b-5091-456c-81ac-253b6459eb87 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.297076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a3464-5442-41d8-97a9-1f5415ca2e64 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.316967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.317804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.318094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.318376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.319130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:07.319471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.320873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.334676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 345/670] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:07.335712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.339646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.342211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.343240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.343948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.344454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.356274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.362833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:07.378345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:07.379290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:07.380494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:07.380494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:07.380494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:07.380494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:07.382787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 326/671] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:07.383994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.435138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.448578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7575e6-eaf1-4a16-a993-33e3c3e1754c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.485910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.486970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.487112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.487464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.493701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f818f6-e999-4dd7-ba96-e7d523662399 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.501409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.503338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:07.527120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:07.528255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:07.529166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:07.529166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:07.529166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:07.529166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:07.531386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 346/672] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:07.533005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.537164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.538342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.538836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.539347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.551916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.553748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:07.578416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:07.579455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:07.580371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:07.580371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:07.580371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:07.580371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:07.580371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 327/673] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:07.580643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.612016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.667892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185902cd-0ce1-44e4-a5eb-2bbebd3cf289 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.694376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.699026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609a2e4-0655-4b8c-924b-5916818b2e7d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.710627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.711776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.712955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.713239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.713973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:07.714286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.715721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.731702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 347/674] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:07.732358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.746468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.747820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.748318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.749062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.750527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:07.751382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.753626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.774436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 328/675] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:07.774436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.871488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ad4a-3d9a-444b-b110-00e714934f2d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.893277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:07.903420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06400023-8fe1-48de-8160-2ce59d5e69b8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:07.912594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.913486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.913923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.914339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.915274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:07.915785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.917349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.930954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 348/676] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:07.932131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:07.959782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:07.960765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:07.961159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:07.961543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:07.974392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:07.977091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:07.998877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:08.000101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:08.001284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:08.003061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 329/677] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:08.004073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.029756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.071150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a3fef-f3e0-477e-9c15-47d66c188847 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.117900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.119588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.120539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.121328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.132891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.135443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.146895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98519e-354a-409b-aa58-0ad84fc9d11d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:08.159207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:08.160191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:08.161081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:08.168868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 349/678] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:08.168868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.200160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.201500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.202229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.203169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.208302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.215779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.217938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:08.239615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:08.240629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:08.241552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:08.249172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 330/679] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:08.249172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.303830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffd766-655e-4399-adc1-ea53e45e1817 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.352194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.353523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.354921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.356315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.357014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.359610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:08.360821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.364466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.383483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 350/680] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:08.384810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.387080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f38770-5ef3-418b-870b-e287c741563f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.427125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.428011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.428447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.428828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.429740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:08.430158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.432541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.451257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 331/681] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:08.451257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.511141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.518059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556459f-94dc-415d-8d9c-ad6d3987a661 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.564762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.565445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.565732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.566014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.566862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:08.567178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.568689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.585761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1950ec-84f6-4676-8ff4-5b66d0bb4adb None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.589760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 351/682] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:08.590742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.632635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.633562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.634002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.634284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.635129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=3f4e0e17040d41bf91cf4022dc4b4d4e) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:08.635527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.637028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccf6f-e08a-47e3-b232-bcad5629ea57 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.650972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 332/683] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/role_assignments?user.id=3f4e0e17040d41bf91cf4022dc4b4d4e&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:08.650972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2949984-f679-4095-b8fa-dbc277d18ed9 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.650972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2949984-f679-4095-b8fa-dbc277d18ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.650972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2949984-f679-4095-b8fa-dbc277d18ed9 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.650972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 333/684] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:08.650972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2949984-f679-4095-b8fa-dbc277d18ed9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:08.666592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.733666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702521f-37bc-4da1-999a-37e4098b54e0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.788079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.789235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.789826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.790342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.791940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=885cc3a3c23c451c9ad7526e7fa83639) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:08.792278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.794479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112769df-49c6-4d0c-8238-b2f093bb9923 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.811794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 352/685] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/role_assignments?user.id=885cc3a3c23c451c9ad7526e7fa83639&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:08.812321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2949984-f679-4095-b8fa-dbc277d18ed9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:08.814205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.814558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.814841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.852629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:08.853875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:08.860214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef767f8-7596-4fe8-99e2-ce6618320980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:08.860214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef767f8-7596-4fe8-99e2-ce6618320980 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:08.860214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef767f8-7596-4fe8-99e2-ce6618320980 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:08.860574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef767f8-7596-4fe8-99e2-ce6618320980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:08.886136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.972812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 353/686] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:08.976688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef767f8-7596-4fe8-99e2-ce6618320980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:08.978550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef767f8-7596-4fe8-99e2-ce6618320980 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:08.981371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:08.998718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.103098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 334/687] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:08:09.104660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3d3fa-b77e-4097-ae31-70e6c1192394 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.104959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3d3fa-b77e-4097-ae31-70e6c1192394 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.105243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3d3fa-b77e-4097-ae31-70e6c1192394 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.106853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 335/688] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:09.116372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3d3fa-b77e-4097-ae31-70e6c1192394 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.133451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7f97c-65a3-4261-930a-4ac6b1464185 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.176621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6935ed8-0988-4949-9225-64732953ca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.177611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935ed8-0988-4949-9225-64732953ca2e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.178000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935ed8-0988-4949-9225-64732953ca2e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.178287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6935ed8-0988-4949-9225-64732953ca2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.191929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6935ed8-0988-4949-9225-64732953ca2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-a6935ed8-0988-4949-9225-64732953ca2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:09.200941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:09.201937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:09.202770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 354/689] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:09.204041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.204229 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.204515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.222532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:09.224031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:09.226292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:09.263825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3d3fa-b77e-4097-ae31-70e6c1192394 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.265772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:09.311198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.311982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.312627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.313260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.325287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:09.326434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 355/690] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:09.327902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.336252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:09.369474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:09.370437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:09.371330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:09.371330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:09.371330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:09.371330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:09.373516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 336/691] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:09.374484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.470535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0aa4-c9d6-4dfc-b088-a3e70f58e6a4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.499899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012fb3-a4c2-42aa-8da4-0c384c46e112 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.515768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.517292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.517777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.518195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.519085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:09.519648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:09.521191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:09.527864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:09.533099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 356/692] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:09.534310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.550975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.553540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.553734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.553963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.570111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:09.573635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:09.600729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:09.602276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:09.603602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:09.603602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:09.603602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:09.603602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:09.606422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 337/693] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:09.607947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.627693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:09.691365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae2c99-3ea5-4117-891a-1ebd4723e89d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.744733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.745881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.746375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.746875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.748009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:09.748620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:09.750230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:09.754181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a09be-2b1b-4c05-9cd5-88e294c684db None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.765693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 357/694] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:09.767506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.790959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.792069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.792519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.792868 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.814261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:09.818012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:09.853339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:09.855409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:09.857507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:09.857507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:09.857507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:09.857507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:09.866611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 338/695] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:09.868444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:09.872984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:09.924729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2aeef-9138-4a81-879c-9215b5b5feca None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:09.964355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:09.965390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:09.965823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:09.966249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:09.967145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:09.967663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:09.969309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:09.981941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:09.984368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 358/696] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:09.985435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.064614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585222d-799e-44f0-8f54-a5ab98da8fa0 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.102341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471654af-4db1-45cc-94cf-6480e35ff4e2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.125148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:10.133037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.134885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.135577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.140604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.144648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.145547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.146025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.146514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.147445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:10.147904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.149489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.151446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.154517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.164892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 359/697] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:10.166021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:10.196569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:10.197573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:10.198508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:10.202359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 339/698] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:10.204330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.283855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e77e7-6eb9-4806-87a4-f09e72f6e43c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.322770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.323888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.324331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.324757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.325623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.327362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.382490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619e131-d21e-449c-93a5-cbdc44215fd8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.401807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:10.418889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.419752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.420041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.420373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.428290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.429743 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.451346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 360/699] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:08:10 2022] POST /identity/v3/users => generated 254 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:10.451346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:10.461711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:10.462761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:10.463730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:10.465878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 340/700] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:08:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:10.478047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.523542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:10.680697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e0a7c-aba8-47bc-9dde-5aa6b6b5e79a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.705316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1c27-c269-4e2e-83e6-47136bd01d72 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.720447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.721324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.721735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.722131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.723111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:10.723572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.725134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.745002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 341/701] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:08:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:10.751333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.758136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.759985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.760415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.760862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.761829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:10.762297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.764064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.780237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 361/702] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:10.787328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:10.792546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:10.867081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ea1eb-a606-4603-9135-8163e4ec76da None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.898137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:10.905824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af854820-a7b5-410d-b1cb-cae3cd9b5f43 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:10.906040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addea815-6d44-4493-856c-28752342ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.912448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addea815-6d44-4493-856c-28752342ab6c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.912448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addea815-6d44-4493-856c-28752342ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.912448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addea815-6d44-4493-856c-28752342ab6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.952047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:10.953090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:10.954051 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:10.954467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:10.955569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=400527de384349f58866dd9e39468307) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:10.956018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.957854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:10.965611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 362/703] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:10 2022] GET /identity/v3/role_assignments?user.id=400527de384349f58866dd9e39468307&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:10.986798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addea815-6d44-4493-856c-28752342ab6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:10.988521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addea815-6d44-4493-856c-28752342ab6c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:11.029276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:11.117357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 342/704] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:08:11.122995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:11.264283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930f28c-18d4-452d-a0f0-4aef0eaedcc1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:11.311763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:11.313347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.313773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.314276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.326490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:11.328417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:11.348869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:11.349990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:11.351060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:11.351060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:11.351060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:11.351060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:11.353960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 363/705] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:11.359887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addea815-6d44-4493-856c-28752342ab6c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:11.478777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addea815-6d44-4493-856c-28752342ab6c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:11.517236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:11.517846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.518136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.518417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.519162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:11.519498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:11.520923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830bbbd-0cb8-4dba-9e16-ec3af208baf0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:11.523918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:11.541575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 343/706] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:11.547167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:11.569774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:11.604201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34553-d7b3-4c95-b250-6d1493145084 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.604463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34553-d7b3-4c95-b250-6d1493145084 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.604734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34553-d7b3-4c95-b250-6d1493145084 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.606571 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 344/707] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:11.615120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb163a3b-ef64-4422-b7a4-ddd41336c1c8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.616263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb163a3b-ef64-4422-b7a4-ddd41336c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.616788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb163a3b-ef64-4422-b7a4-ddd41336c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.642241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-cb163a3b-ef64-4422-b7a4-ddd41336c1c8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:11.645366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cb163a3b-ef64-4422-b7a4-ddd41336c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:11.717558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dca0a8-09be-45c9-9bb5-91d301ade3ff None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:11.751680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 345/708] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:08:11.751680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbac21-81a2-4062-bebe-7812e5f42caf None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.751680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbac21-81a2-4062-bebe-7812e5f42caf None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.751680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbac21-81a2-4062-bebe-7812e5f42caf None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.755631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 346/709] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:11.758608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.759097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.759621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.781585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:11.783415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:11.783954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:11.784447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:11.792660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:11.793935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:11.795544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:11.798952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:11.819471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:11.820477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:11.821415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:11.829471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 364/710] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:11.829471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:11.908039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 347/711] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 01 04:08:11.909006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:11.938937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:11.987633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:11.988543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeec931-7ba0-4239-930a-0127fb7b63a8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.038417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.039189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.039503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.039821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.040490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.041888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.044070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655ff2-e5d3-40fe-81f0-abe5149fb4a1 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:12.057318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:12.058343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:12.060239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 365/712] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Jun 1 04:08:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:12.061099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:12.102303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.103637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.104181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.104675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.106352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:12.106897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.109217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.130990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 348/713] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:12.130990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:12.187362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafd594-0f2b-43af-9632-b5a59f01573e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.228218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.230186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.231240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.232222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] PATH_INFO: `/v3/services/network` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.234047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.237833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.242039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775e12-ffeb-438b-8f0b-9f1da338769a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.244218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:12.260176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:12.263471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:12.264327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:12.265118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:12.266146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 366/714] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:12 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:12.267373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:12.296823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.298896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.299443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.299940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.301089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:12.301601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.303965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.315692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 349/715] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jun 1 04:08:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:12.315948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:12.413487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c374a-d9a4-4ad1-807e-1ea172de7fad None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.448276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c93db4-dbec-436d-a96f-48401db920b8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.449801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.450577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.450794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.451016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.451894 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=56036136066d412dac682273ebf65ab9) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:12.452193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.454183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.467332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 367/716] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:12 2022] GET /identity/v3/role_assignments?user.id=56036136066d412dac682273ebf65ab9&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:12.467332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:12.514783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.516590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.517182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.517678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.518713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:12.519293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.521180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.532435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:12.536672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 350/717] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:12.543253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:12.582392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503be819-4919-4a07-b8d0-47584236c2d8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.621794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.621794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.621794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.621794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.631648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:12.631648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.631648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd57fa-4f00-44a6-a0a4-7a5a3de78b10 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.632751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 368/718] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jun 1 04:08:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:12.661508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:12.748239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32ca5-349c-481c-a6c5-1133f2256ad8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:12.807473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:12.809160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:12.809692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:12.810195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:12.811194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] RBAC: Adding query filter params (type=network) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:12.811777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:12.814059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f2c8c-d060-43d8-a77f-7acf5b356798 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:12.823361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 351/719] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:13.740962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07efe7-81b4-4243-bd98-7c0a0dd7bc30 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:13.741734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07efe7-81b4-4243-bd98-7c0a0dd7bc30 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:13.741734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07efe7-81b4-4243-bd98-7c0a0dd7bc30 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:13.743122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 369/720] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:13.761316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:13.761721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:13.761966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:13.784339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:13.785560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:13.876690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 352/721] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:13.914445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07efe7-81b4-4243-bd98-7c0a0dd7bc30 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.073989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07efe7-81b4-4243-bd98-7c0a0dd7bc30 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:14.111981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:14.122750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bba32e-cab9-4959-8d65-da5acf8a7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:14.132551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba32e-cab9-4959-8d65-da5acf8a7f08 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.132551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba32e-cab9-4959-8d65-da5acf8a7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.132551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba32e-cab9-4959-8d65-da5acf8a7f08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.136245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bba32e-cab9-4959-8d65-da5acf8a7f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-68bba32e-cab9-4959-8d65-da5acf8a7f08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:14.148361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:14.150165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:14.153739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 370/722] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:14.156337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.211773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:14.274903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c7335-f681-447a-9d30-0e0195259148 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:14.334399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce803d-fd7d-4737-8e16-3cefacb3e82d None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.334742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce803d-fd7d-4737-8e16-3cefacb3e82d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.335036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce803d-fd7d-4737-8e16-3cefacb3e82d None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.336587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 371/723] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:14.340293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:14.340908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.341198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.341499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.342411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:14.342788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:14.344411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:14.357593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 353/724] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:14.359168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.359742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.360231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.372602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.392553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:14.394136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:14.489710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:14.495470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227bdd8-12dc-47a1-98e6-0f70b34e67bd None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:14.524588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 372/725] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:14.554832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:14.555675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.555970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.556295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.568481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:14.570097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:14.596795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:14.597825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:14.598703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:14.598703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:14.598703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:14.598703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:14.598703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 354/726] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:14.605313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.605774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.724490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c2351-f60d-4798-a09a-a09a729ca5d3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:14.776992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:14.785767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f29ee-9969-4e6d-bc1f-f9f06ff21914 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:14.810497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:14.811709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.812010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.812295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.823752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:14.824471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:14.824754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:14.825075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:14.825818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:14.826129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:14.827608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:14.837145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:14.840810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 355/727] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:14.845769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:14.851810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:14.855401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:14.858768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 373/728] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:14.870386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:14.879669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:15.039053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1825c-c344-4ef3-8859-9fc06a602eae None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:15.092130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255733b1-f32b-4151-bbb1-ec94b426ee72 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:15.114369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:15.115205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.115748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.116041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.129758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:15.131527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:15.151936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:15.155724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:15.160130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 356/729] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:08:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:15.168572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:15.200093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:15.200793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.201059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.201348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.202128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:15.202524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:15.204089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:15.223548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 374/730] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:15.245659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:15.257322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:15.301187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58509b5a-5230-4fa1-ad2c-36b80595b653 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:15.389658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:15.389916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:15.390907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.391482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.391997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.393696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:15.394337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:15.397230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:15.452416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 357/731] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jun 1 04:08:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:15.456627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ceb270-7f1b-48b1-9a53-ec8a2de5e9b9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:15.458961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:15.502891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:15.503757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.504043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.504323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.526516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:15.528920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:15.552875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:15.553807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:15.554710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:15.554710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:15.554710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:15.554710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:15.554710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 375/732] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:15.568638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:15.685823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76f99-657b-4619-af8e-612675e235c3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:15.743906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:15.754394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6e2d4-0327-4079-8ed6-2d970d740b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:15.755858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6e2d4-0327-4079-8ed6-2d970d740b9c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.756525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6e2d4-0327-4079-8ed6-2d970d740b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.763975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6e2d4-0327-4079-8ed6-2d970d740b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.776257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:15.794947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd88ca-516a-4618-b179-932a8f0fe90a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:15.840680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:15.841274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.841557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.841834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.842673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:15.842985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:15.844490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244ee85-d3c0-4624-8a81-932f87e9c2af None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:15.858299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 376/733] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:15.859876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a5b7a-427c-48fc-b6ad-710d4705e9d4 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.860286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a5b7a-427c-48fc-b6ad-710d4705e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.860579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a5b7a-427c-48fc-b6ad-710d4705e9d4 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.862253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 377/734] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:15.870535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778946-3840-471e-96d9-248bf508711d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:15.870833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778946-3840-471e-96d9-248bf508711d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:15.871227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778946-3840-471e-96d9-248bf508711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:15.891910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-7c778946-3840-471e-96d9-248bf508711d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:15.892113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6e2d4-0327-4079-8ed6-2d970d740b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:15.893164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7c778946-3840-471e-96d9-248bf508711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:15.895024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6e2d4-0327-4079-8ed6-2d970d740b9c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:15.979363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 378/735] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:15.979363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c778946-3840-471e-96d9-248bf508711d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.077045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 358/736] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 01 04:08:16.079999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f4d06-32f0-4d0b-9c61-bf0eb5c9056e None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.080516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f4d06-32f0-4d0b-9c61-bf0eb5c9056e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.080997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f4d06-32f0-4d0b-9c61-bf0eb5c9056e None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.082482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.083905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 359/737] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:16.085272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f4d06-32f0-4d0b-9c61-bf0eb5c9056e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.108501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c778946-3840-471e-96d9-248bf508711d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.138225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.149547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.150935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.151488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.152014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.165861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:16.167756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:16.185234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:16.186253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:16.187197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:16.194044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 379/738] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:16.194044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.237342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f4d06-32f0-4d0b-9c61-bf0eb5c9056e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.271554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96c48e-3663-4118-a89f-0931c5f4e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.272356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96c48e-3663-4118-a89f-0931c5f4e6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.272634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96c48e-3663-4118-a89f-0931c5f4e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.272905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96c48e-3663-4118-a89f-0931c5f4e6a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.283477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96c48e-3663-4118-a89f-0931c5f4e6a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ff96c48e-3663-4118-a89f-0931c5f4e6a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:16.291397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:16.292604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:16.293437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 360/739] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:16.294211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adad3ff-1bc2-4cbc-9797-c87700a03dde None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.294211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adad3ff-1bc2-4cbc-9797-c87700a03dde None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.294211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adad3ff-1bc2-4cbc-9797-c87700a03dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.309929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0887-d154-4429-9253-c52bdd5b39f8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.332243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-7adad3ff-1bc2-4cbc-9797-c87700a03dde None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:16.334101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7adad3ff-1bc2-4cbc-9797-c87700a03dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:16.353093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.353907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.354177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.354460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.366490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:16.367984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:16.370023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:16.387995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:16.389076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:16.389970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:16.389970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:16.389970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:16.389970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:16.389970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 380/740] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:16.390508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.418505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 361/741] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:16.420162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4fd24-a9c7-4cc1-9415-01bd7edfd373 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.420596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4fd24-a9c7-4cc1-9415-01bd7edfd373 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.420917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4fd24-a9c7-4cc1-9415-01bd7edfd373 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.423167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 362/742] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:16.427866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4fd24-a9c7-4cc1-9415-01bd7edfd373 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.453497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.517383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a18fb-6125-4869-b302-24d9359da5e8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.552877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4fd24-a9c7-4cc1-9415-01bd7edfd373 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.555255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.555974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.556260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.556541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.557381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:16.557694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:16.559152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:16.577355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 381/743] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:16.578227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.597614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.598715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.599647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.599799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.600712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:16.601123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:16.602541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c16a-21fa-4612-a3e8-3023f8004847 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:16.612237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 363/744] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:16.613232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.613518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.613795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.635054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:16.636353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:16.644784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.698250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee44e43-34b0-479a-82a2-097bc7e6b4e2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.719681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 364/745] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 01 04:08:16.721209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.737492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.738150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.738743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.738873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.739937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:16.740064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:16.741542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:16.741708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.753755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 382/746] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:16.754628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:16.832449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e4997-4c01-4999-a369-2ee55d5220d2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.869506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.870277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.870627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.870966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.871810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:16.873262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:16.903935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80b977-42ed-42b4-88f4-2512fff6173a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:16.903935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:16.921565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:16.922798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:16.923081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:16.923380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:16.994016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 365/747] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:16 2022] POST /identity/v3/services => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:16.994874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.001863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.003454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.116285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:17.161315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 383/748] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 01 04:08:17.161315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.180713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:17.192597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e881c8-688d-438c-8693-f816f48f9c81 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.239897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.240710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.240998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.241277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.254513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.256523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:17.281113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:17.282633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:17.283853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:17.283853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:17.283853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:17.283853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:17.283853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 366/749] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:17.283853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.294728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f4051-e754-46c7-9949-759c77e767a5 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.337859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.338528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.338808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.339082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.339840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.341274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.397432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d5d05-a75e-4b83-8d7a-8e841ec1eb51 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.419494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 384/750] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:17 2022] POST /identity/v3/services => generated 240 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:17.421158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.430465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:17.450106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.451183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.451561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.452270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.460757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.462488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:17.480746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:17.482861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 367/751] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:17.485575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.548215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:17.595484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2287f0-8d63-47db-aa7a-12e096f922a7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.650227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.652740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.653219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.653733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.667642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.668855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b3a5f-8084-455a-b7f8-6be480460308 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.669191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:17.690392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:17.691380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:17.692290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:17.692290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:17.692290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:17.692290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:17.692290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 385/752] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:17.692559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.720272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.720962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.721255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.721539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.722280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:17.723162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.724087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.744516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 368/753] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:17.745632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.785214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:17.840047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a59549-6413-4652-af8a-2d918bab9d43 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.870697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25910-80e1-4cbd-9e7c-46290134c467 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:17.872135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:17.884586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.885492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.885878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.886275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.887487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:17.887866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.889638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.904792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 386/754] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:17.905889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:17.918762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:17.919749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:17.920559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:17.921018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:17.922290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:17.922754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:17.924623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:17.940801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 369/755] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:17.948262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.045153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37ba0e-a1e8-42b4-9c59-e6c1f78622cd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:18.078031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a02dc6-5430-4a8d-aa8a-981a1c68644f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:18.081367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:18.099646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:18.101401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:18.101888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:18.102171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:18.115567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:18.118329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:18.118639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:18.119570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:18.120051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:18.120542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:18.122565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=c58ac60d4c7441a4ba1053cbbb14b907) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:18.123110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:18.124891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:18.132742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 370/756] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/role_assignments?user.id=c58ac60d4c7441a4ba1053cbbb14b907&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:18.134035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:18.135695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:18.136712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:18.137686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 387/757] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:08:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:18.147114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.242122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:18.344583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcd1ea-bb2b-4b1a-8804-e4c3ed776023 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:18.386671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8569e-9e19-419f-81c5-a395fcd39e1c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:18.405905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:18.406991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:18.407505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:18.408287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:18.409335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:18.409883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:18.411675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:18.431911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 388/758] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:08:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:18.439103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.454470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908bf75-9779-4261-baed-4df78222287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:18.455738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908bf75-9779-4261-baed-4df78222287b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:18.456547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908bf75-9779-4261-baed-4df78222287b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:18.458297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908bf75-9779-4261-baed-4df78222287b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:18.473132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908bf75-9779-4261-baed-4df78222287b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:18.475265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908bf75-9779-4261-baed-4df78222287b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-6908bf75-9779-4261-baed-4df78222287b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:18.509908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:18.510850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:18.511782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:18.517737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 371/759] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:18.524270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908bf75-9779-4261-baed-4df78222287b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.566196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:18.657180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4afd-0d48-4e33-886a-fd4ff0779b7d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:18.658929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:18.678262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908bf75-9779-4261-baed-4df78222287b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:18.735886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:18.737064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:18.741084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:18.741753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:18.742913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:18.743466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:18.745168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:18.760666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:18.762079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:18.762562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:18.763039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:18.779522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 372/760] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:18.788454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.793711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:18.795342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:18.814858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:18.816706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:18.818511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:18.818511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:18.818511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:18.818511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:18.818511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 389/761] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:18.828445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:18.983677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:19.025715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2268681-570a-48f5-9d90-98e912075ed4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:19.068378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08f4ed-431d-4d21-9476-da57ee3ff6e6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:19.112441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:19.113137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:19.113395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:19.113724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:19.114577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=400527de384349f58866dd9e39468307) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:19.114946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:19.116511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:19.135834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 373/762] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:18 2022] GET /identity/v3/role_assignments?user.id=400527de384349f58866dd9e39468307&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:19.155251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:19.156001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:19.156826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:19.157125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:19.157880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:19.158188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:19.159684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:19.173410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 390/763] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:19.182719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:19.422577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bf846-e5dc-49cf-80d7-db2fd0bfc2c4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:19.510570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:19.511844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:19.512342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:19.512856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:19.528679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:19.530169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:19.552545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:19.553548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:19.554858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 374/764] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:19.561222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:19.591122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:19.730859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79c189-7924-4a9b-b335-3ac2a8dddbf7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:19.789815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:19.790875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:19.791404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:19.791888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:19.792913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:19.793430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:19.795444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130443-4087-49a9-983b-08b232371bd5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:19.827752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 391/765] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:19.837109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:19.853438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:20.037793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a51b1-33e9-4667-8ee6-9e0c1ff82ada None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:20.092199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd73fa-f8de-4669-a087-bfda105501d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:20.093106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd73fa-f8de-4669-a087-bfda105501d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.093397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd73fa-f8de-4669-a087-bfda105501d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.093700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd73fa-f8de-4669-a087-bfda105501d0 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/56036136066d412dac682273ebf65ab9/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.100766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd73fa-f8de-4669-a087-bfda105501d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=56036136066d412dac682273ebf65ab9, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:20.102368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd73fa-f8de-4669-a087-bfda105501d0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:20.154042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879db6e-21ce-4523-bb8a-00ea45be8b99 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.155660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879db6e-21ce-4523-bb8a-00ea45be8b99 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.156470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879db6e-21ce-4523-bb8a-00ea45be8b99 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.166099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 392/766] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:20.169153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.169965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.170786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.199499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:20.206954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:20.211518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 375/767] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:19 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/56036136066d412dac682273ebf65ab9/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:20.265012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85c902-5a64-44f7-bdd8-3eaa7b42b587 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.265701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85c902-5a64-44f7-bdd8-3eaa7b42b587 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.266223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85c902-5a64-44f7-bdd8-3eaa7b42b587 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.268015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 376/768] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:20.274071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.274323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.274660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.307147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:20.309936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:20.350444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 393/769] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:20.409624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:20.414165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:20.466720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:20.483392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 377/770] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:20.512808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:20.597174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfc6d4-b564-43f8-94dd-72d956d3bebd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:20.649091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d493dd-a6c3-41db-bed6-72d03cdbbb3c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:20.656369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:20.657262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.657684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.657973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.658658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:20.660278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:20.685377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:20.686628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:20.688473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 394/771] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Jun 1 04:08:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:20.696310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:20.704866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:20.706130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.706663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.707146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] PATH_INFO: `/v3/services/network` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.708124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:20.709796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:20.739354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:20.740444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:20.741308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:20.752036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 378/772] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:20 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:20.753961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:20.861473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:20.901485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:20.925857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d1ff5-f536-4013-8eeb-33e97063e57f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:20.975373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:20.976066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:20.976357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:20.976638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:20.977353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:20.977668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:20.979084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:20.990695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 395/773] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jun 1 04:08:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:20.997966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.002317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e366fe-22bf-42c3-a7fc-9bfe78a0f62e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.089681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.090738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.091227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.091747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.092707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] RBAC: Adding query filter params (name=network) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:21.093222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.095010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.106034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 379/774] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:21.113976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.196165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d607a-6abf-4d89-8090-206426fd633a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.237685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:21.243251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.244456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.244980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.245478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.246676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:21.247189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.248817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.257066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 396/775] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jun 1 04:08:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:21.264319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.339015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c30ec1-21ae-4671-b2cc-6ad65538ba90 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.363624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:21.377657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.378261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.378658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.379049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.380236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] RBAC: Adding query filter params (type=network) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:21.380645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.382243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8d914-b5dc-4d31-88c8-4202136e8125 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.389649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 380/776] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:21.392179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e3af-97d0-4cbc-bcfe-290020c8b191 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.392713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e3af-97d0-4cbc-bcfe-290020c8b191 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.393235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0e3af-97d0-4cbc-bcfe-290020c8b191 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.395914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 381/777] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:21.397473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e3af-97d0-4cbc-bcfe-290020c8b191 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.414841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6205d4-98b8-42ce-a48a-4c06d409a43a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.453211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.454206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.454714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.455211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.456566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] RBAC: Adding query filter params (interface=public, service_id=027b4d5fd0fc4db695f0816367c1afe5, region_id=RegionOne) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:21.457130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.459816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8376dc6-d8ad-4b52-a0b8-cef23a395ecc None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.471273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 397/778] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/endpoints?service_id=027b4d5fd0fc4db695f0816367c1afe5&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:21.472687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.472980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.473260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.493633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:21.495642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:21.532839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0e3af-97d0-4cbc-bcfe-290020c8b191 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.571186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.572026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.572381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.572732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.573635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] RBAC: Adding query filter params (interface=public, service_id=d19deed1a34e43d092f44089f6a8a094, region_id=RegionOne) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:21.574014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.575644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.584711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 382/779] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/endpoints?service_id=d19deed1a34e43d092f44089f6a8a094&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:21.585689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.599912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:21.619506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 398/780] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:21.620531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.650957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:21.704971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce19fd1-866d-4de2-9d5f-c2552a386e54 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.747734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.748905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.749365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.749799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.751251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.753094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.760874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 383/781] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:21.762162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:21.820445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715a563-6126-425b-9f70-51e93dbc2b72 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.895610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32134343-0a80-499e-aed7-d167e5f1f5c9 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:21.914086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:21.940634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843777-fb55-40e2-8a7f-ae859c80a3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.941848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843777-fb55-40e2-8a7f-ae859c80a3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.942342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843777-fb55-40e2-8a7f-ae859c80a3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.942825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843777-fb55-40e2-8a7f-ae859c80a3e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.949213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:21.949981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:21.950185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:21.950465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:21.951257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:21.952736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:21.958275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843777-fb55-40e2-8a7f-ae859c80a3e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-3b843777-fb55-40e2-8a7f-ae859c80a3e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:21.967728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:21.969284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:21.970159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:21.970801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 399/782] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/services => generated 772 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:21.973328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 384/783] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:21.980407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:22.097200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:22.160892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834972a1-6f9d-491d-9118-3d1482f9c525 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:22.178602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0910-7fae-4350-ab55-95b9ad6762b7 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:22.179236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0910-7fae-4350-ab55-95b9ad6762b7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:22.179754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e0910-7fae-4350-ab55-95b9ad6762b7 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:22.181791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 385/784] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:22.192061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:22.193219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:22.194609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:22.221940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:22.222692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:22.222981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:22.223386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:22.224439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:22.224439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:22.225947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:22.236065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 400/785] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:22.236863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:22.238103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:22.242718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:22.370059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 386/786] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:22.412504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:22.433774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4915bd-8126-4bac-a95a-7ad6c3ae100f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:22.454221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:22.484528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:22.486488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:22.487341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:22.487833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:22.533644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:22.539520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:22.602416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:22.603592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:22.604523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:22.604523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:22.604523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:22.604523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:22.609506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 401/787] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:22.624503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:22.741252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9ba1e-1428-457c-9b5f-6f6b5709143d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:22.752638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:22.781631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e0e9-5727-4947-a96b-79adc9a8c5a0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:22.815550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:22.816381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:22.816681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:22.816986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:22.824241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:22.827681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:22.829212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:22.830517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:22.831501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:22.833302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:22.834925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:22.842346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:22.859723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:22.860738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:22.861610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:22.867737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 402/788] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:22.869132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 387/789] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:22.876066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:22.876288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:22.916694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:23.011147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81f808-3fc8-4708-91eb-50d4a6f604f1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.049521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.050350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.050635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.050914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.059717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.061210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:23.078595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:23.079735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:23.080697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 403/790] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:08:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:23.086281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:23.093916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b28cb2-42b7-44af-97f0-5bb45e6b2a33 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.200097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:23.215029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738de97-9fb2-46d0-938e-b20b4187742b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.216459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.217594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.218083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.218606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.219814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:23.220320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.223062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.251965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 388/791] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:23.260564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.261299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.261539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.261891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.262816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:23.263129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.264984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.271480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:23.290977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 404/792] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:23.292969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:23.317186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:23.411971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40b07c-4a6c-407c-8b84-2f9e802483da None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.450428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.451232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.451536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.451810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.462883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.464464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.472296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:23.487611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:23.488574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:23.489474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:23.489474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:23.489474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:23.489474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:23.494010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 405/793] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:23.495448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:23.544033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccea8-444c-4d96-8a60-6cb892a443c3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.614080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.615494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.615989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.616482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.638674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.640066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-c4d29068-53c5-4b4b-a3d2-463fc886c515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:23.659342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:23.660320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:23.661234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:23.661234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:23.661234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:23.661234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:23.665273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 389/794] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:23.666887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d17c7-eb11-4f84-9354-1b9d2f120b58 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.667183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d17c7-eb11-4f84-9354-1b9d2f120b58 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.667489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d17c7-eb11-4f84-9354-1b9d2f120b58 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.669420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 390/795] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:23.670719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d17c7-eb11-4f84-9354-1b9d2f120b58 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:23.674770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a62cc-80b0-41a2-b1b0-051ad6497045 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.723082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.723935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.724358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.724673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.725811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:23.726163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.727688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49fc97-9324-4798-b233-64d1f3af751d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.745238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 406/796] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:23.747370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.747672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.747950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.769022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:23.770748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:23.791946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:23.813998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d17c7-eb11-4f84-9354-1b9d2f120b58 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:23.853288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:23.869670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 407/797] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:23.870843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:23.894030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:23.895332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:23.895925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:23.896502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:23.898123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:23.898752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:23.901507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:23.937887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 391/798] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:23.944693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.016232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51d6e0-5fc5-45f3-8e7e-a79efc242301 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.061396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.063253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.063753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.064266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.073379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.075084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:24.095421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:24.096391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:24.097638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:24.101853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 408/799] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:24.102659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.132681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748404c2-de1b-4fd1-801c-191b313eb1be None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.175691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:24.195869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.197260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.197778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.198317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.212557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:24.221132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:24.221468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c7fa3-518c-4b22-a017-cbffc1cce9ef None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:24.237306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:24.238318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:24.239153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:24.240706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 392/800] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:24.242402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.282695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.283980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.284363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.285169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.296196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.297891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:24.320288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:24.321313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:24.322275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:24.327662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 409/801] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:24.329255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.370837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3418a6f-0e2b-4a21-899b-54385260c0e4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.410224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.411049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.411437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.411791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.412671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:24.413057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.414626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.433295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 393/802] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:24.434393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.457648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efa2e0-4fb3-4ece-83b5-f454a0d08e0b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.475714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:24.496397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.497123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.497412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.497689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.498440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:24.498744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.500301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.506858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:24.510979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 410/803] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:24.511911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.553185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94da4b-5e68-4c93-8f0b-35bd54518827 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.605679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.606303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.606595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.606878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.607795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:24.608107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.610749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.649004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 394/804] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:24.649004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.664654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19429a5-c2a9-4660-9d4e-7bae778ceda3 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.712710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.714679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.715353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.715959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/c58ac60d4c7441a4ba1053cbbb14b907/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.740923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=c58ac60d4c7441a4ba1053cbbb14b907, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.743149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.803043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f312d6-4ec6-416e-b6f3-85de0aff6d08 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:24.831612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:24.841445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 411/805] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:24 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/c58ac60d4c7441a4ba1053cbbb14b907/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:24.842860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:24.850446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:24.851431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.851821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.852270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.875420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:24.877097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:24.891414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-90ff606d-a155-4baa-badf-577e5c116257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:24.914829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:24.916592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:24.918364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:24.918364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:24.918364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:24.918364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:24.918364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 395/806] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:24.922129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6c69-23ea-429f-9ea3-1d007f785464 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:24.922626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6c69-23ea-429f-9ea3-1d007f785464 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:24.923580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6c69-23ea-429f-9ea3-1d007f785464 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:24.928006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 396/807] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:24.929348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6c69-23ea-429f-9ea3-1d007f785464 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.028346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a266a-f218-4cc2-8124-817a426a40d8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.090682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6c69-23ea-429f-9ea3-1d007f785464 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.096964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.097798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.098077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.098349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.117641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.118989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-48b084c1-0ba7-4c75-aa46-8c1398c52fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:25.145433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:25.146366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:25.147247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:25.147247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:25.147247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:25.147247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:25.147247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 412/808] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:25.148112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.148112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.148386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.152320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.153187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.153485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.153772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.154904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:25.159431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.159431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.168395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:25.169857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:25.176049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 397/809] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:25.180223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.212103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:25.280903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 413/810] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:25.282745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.289606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:25.323383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735d930-153b-4db1-829b-24b52bdc52ff None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.366111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.367222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.367731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.368166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.369329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:25.369836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.372024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3feaad-7467-41bf-b668-6c0ddf2ef2e5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.389289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 398/811] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:25.391005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33159325-c2d6-4de4-ac67-8d9d086f044f None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.391528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33159325-c2d6-4de4-ac67-8d9d086f044f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.392087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33159325-c2d6-4de4-ac67-8d9d086f044f None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.393645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 399/812] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:25.395073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33159325-c2d6-4de4-ac67-8d9d086f044f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.443403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527bdf7-713c-4b65-8fbb-0c74d6952a8b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.486841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.487841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.488226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.488566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.497230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.498724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:25.519360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:25.520996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 414/813] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:25.522785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.549158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:25.564453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33159325-c2d6-4de4-ac67-8d9d086f044f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.600449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:25.616671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.617574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.617970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.618392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.619540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.621974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-625cd8ab-aabc-4313-b87f-17a6d83fd0c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:25.641152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:25.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 01 04:08:25.643022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:25.647429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 400/814] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:25.649002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.649726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.649726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.668764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd4273-ba1a-4879-8f63-434e2c269ee4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.671511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:25.673298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:25.712282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.713031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.713376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.713661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.727214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.730348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:25.754420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:25.755812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:25.756709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 415/815] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:25.757814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.783407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 401/816] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:25.783407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:25.908612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:25.919599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346040d5-ae24-42dd-ad49-91d2f968a23a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.934577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a1c80-4756-4e60-a63f-76005822ca05 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:25.951025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:25.965003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.965936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.966223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.966481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.967965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:25.968284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.970752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.979112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:25.979834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:25.980114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:25.980429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:25.981299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:25.981609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:25.982986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:25.988706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 402/817] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:25.989744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.007055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 416/818] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:26.008469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.133188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9826b-6a90-4983-b8fd-19bf97c3409a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.147907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3088b2-dd79-4f1e-99d4-44f0c940fa18 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.183369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.183369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.183369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.183369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.183996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:26.183996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.184696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:26.208977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 403/819] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:26.208977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.212667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.213486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.213768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.214814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] PATH_INFO: `/v3/services/network` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.214814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.216162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:26.222839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:26.231395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:26.236330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:26.237177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 417/820] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:26 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:26.237968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.304630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:26.373619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e62bd5-ce4f-497a-967b-19db971d3f78 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.383867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2c7a-e768-4ad1-9ab7-1e71da835303 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.424582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.425601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.425881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.426043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.426425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.426976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.427160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:26.427610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.427741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.428175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.429074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:26.443077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 404/821] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jun 1 04:08:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:26.443077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.520214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.521604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:26.530340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:26.576780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02e52-8817-4a8f-8bc7-63a6160bc050 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.589131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:26.636412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.637741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.638036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.638312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/400527de384349f58866dd9e39468307/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.641341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=400527de384349f58866dd9e39468307, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.643621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:26.665210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 418/822] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:08:26.666181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.719338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 405/823] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:26 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/400527de384349f58866dd9e39468307/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:26.720273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.833825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4a471-21b8-487d-8624-86b6080c47f7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.894411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8b97f-57a3-4e64-adad-1000b041574c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:26.894722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.895756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.896637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.897161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.899281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] RBAC: Adding query filter params (name=network) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:26.899923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.901587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:26.910691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 419/824] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:26.912202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:26.932268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:26.942272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:26.951296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:26.951832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:26.952103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:26.952388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:26.953193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:26.954617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:27.059813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 406/825] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:27.063332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:27.096994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19561cf6-8eb1-4638-ac40-661392614591 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:27.187340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:27.190093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.191000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.192286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.216930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:27.219368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:27.252638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:27.253637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:27.254849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:27.254849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:27.254849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:27.254849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:27.266594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 420/826] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:27.271134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:27.299195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1822b6f-751a-4267-b2eb-0e9740814d36 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:27.349012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:27.354951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:27.355928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.356450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.356872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.357740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:27.358193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:27.363288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:27.368812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 407/827] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:27.376422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:27.412506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1302d89-3d06-4cf2-a152-9e72973bb375 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:27.451110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:27.456354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6002381-3319-4880-b539-764e32121683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:27.460352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6002381-3319-4880-b539-764e32121683 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.460352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6002381-3319-4880-b539-764e32121683 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.460352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6002381-3319-4880-b539-764e32121683 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.460352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6002381-3319-4880-b539-764e32121683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:27.460352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6002381-3319-4880-b539-764e32121683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:27.460352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6002381-3319-4880-b539-764e32121683 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:27.477535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 421/828] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:27.479362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97f85f-7502-4c00-8405-e94a411abed6 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.479869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97f85f-7502-4c00-8405-e94a411abed6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.480402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97f85f-7502-4c00-8405-e94a411abed6 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.482249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 422/829] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:27.490872 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97f85f-7502-4c00-8405-e94a411abed6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:27.540857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f317-f8bd-4fa9-b218-37bc6b36e6c2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:27.590043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c217-e915-4f05-a161-104905df3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:27.590834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c217-e915-4f05-a161-104905df3d9f None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.591057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c217-e915-4f05-a161-104905df3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.591421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c217-e915-4f05-a161-104905df3d9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.592203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c217-e915-4f05-a161-104905df3d9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:27.594169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c217-e915-4f05-a161-104905df3d9f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:27.634502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97f85f-7502-4c00-8405-e94a411abed6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:27.655621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:27.691877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 408/830] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:27.693540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.693927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.694379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.698112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:27.699387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.699960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.701672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.713230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:27.716480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:27.731684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:27.732900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:27.739238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:27.740352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:27.741251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:27.744174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 423/831] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:27.753010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:27.913368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 409/832] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:27.942114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76f0c3-fcc5-4213-9800-c3c1b4da28d9 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:27.965162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:27.983946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:27.984638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:27.984930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:27.985207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:27.986003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:27.986312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:27.987797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:28.003928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 424/833] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:28.012417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:28.018440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:28.061335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:28.110237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0152-9e6d-4b21-bba5-90f353e99939 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:28.146662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40caadec-183d-4c5c-8126-1c7ea47a3854 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:28.164448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:28.165286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:28.165578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:28.165857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:28.178602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:28.182385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:28.183046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:28.183352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:28.183633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:28.184470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=56036136066d412dac682273ebf65ab9) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:28.184778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:28.186211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:28.187781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:28.188800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:28.190127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 410/834] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:08:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:28.195712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:28.196202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 425/835] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:28 2022] GET /identity/v3/role_assignments?user.id=56036136066d412dac682273ebf65ab9&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:28.320762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:28.326591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd19c9-dc4d-4368-bb86-1c18369fed25 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:28.364159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:28.364841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:28.365124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:28.365401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:28.366153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:28.366461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:28.368305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:28.381405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 411/836] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:08:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:28.386953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:28.492987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:28.498573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a863d-0050-44fc-bdca-7b036c76422b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:28.534895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:28.535707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:28.535992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:28.536273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:28.548565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:28.550037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:28.568219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:28.569186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:28.570299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:28.570299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:28.570299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:28.570299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:28.570299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 426/837] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:28.575616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:28.696790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222416a8-0b37-4f5b-9d7f-81a0367af01c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:28.736288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:28.737338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:28.737841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:28.738323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:28.739254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:28.739799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:28.741384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:28.754183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 412/838] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:28.759556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:28.866124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a77893-0b5a-4f7b-bee3-531b9c66bf07 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:28.906768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:28.907619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:28.907907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:28.908182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:28.915961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:28.917402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:28.932301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:28.933275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:28.934240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 427/839] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:08:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:28.939294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:28.958605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.040232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0a54-6813-4423-b7f2-817c2a914e8b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.074686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.075385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.075669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.075945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.076778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:29.077085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:29.079037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5dd15-712b-464f-89c9-4ea5d0f5e7b4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:29.097572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 413/840] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jun 1 04:08:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:29.103067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.119709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.208184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9a04d-d605-4ae9-9aa8-8009835622f8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.221794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7d7a8-a1d8-47c0-bdcf-890182a53393 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.222113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7d7a8-a1d8-47c0-bdcf-890182a53393 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.222371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7d7a8-a1d8-47c0-bdcf-890182a53393 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.224026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 414/841] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:29.229311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.229602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.229886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.246373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.247400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.247901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.248393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.249780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:29.250965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:29.327175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:29.329761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:29.341103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.354645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 415/842] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:29.377763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.386504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.453747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 428/843] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:08:29.455461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.486766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f610d-235b-44cd-8471-885e7f595926 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.524496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.525322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.525609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.525943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.537955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:29.545319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:29.546252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:08:29.547773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:29.551425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 416/844] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:29.553224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.573826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f4051-4fd7-407e-88fa-15ed302549b8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.584614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.612580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.613640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.614132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.614478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.624727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.637706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:29.644126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:29.659332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:29.660365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:29.661622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:29.661622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:29.661622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:29.661622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:29.661622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 429/845] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:29.665402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.666102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae45a9-55da-431a-9076-753333090c38 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.703255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.704141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.704523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.704891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.705759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:29.706163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:29.707694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:29.717130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 417/846] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:29.724161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.789156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a3aa6-9f58-4ce5-b2cf-3956d7ff39b6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.827401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.828566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.828847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.829177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.830034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:29.830346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:29.831870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9da5-49c7-42a5-9628-9e9798cd1cd4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:29.851927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 430/847] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:29.853205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911d147-7ce5-43e9-9552-d8b2f77f72c1 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.853556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911d147-7ce5-43e9-9552-d8b2f77f72c1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.853846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911d147-7ce5-43e9-9552-d8b2f77f72c1 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.855356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 431/848] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:29.856604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199d40-3536-4bd9-adce-d5daca43b36a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:29.856802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911d147-7ce5-43e9-9552-d8b2f77f72c1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:29.880454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.885123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:29.894464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:29.895262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.895580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.895856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.907640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:29.909090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-facec7e6-7d9d-49b1-b8a4-55d1acc23c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:29.926049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:29.926972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:29.928255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:29.928255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:29.928255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:29.928255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:29.928255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 418/849] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:29.928792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b815-894e-4e2f-918c-31236156762d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:29.929051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b815-894e-4e2f-918c-31236156762d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:29.929383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b815-894e-4e2f-918c-31236156762d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:29.946730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fd21b815-894e-4e2f-918c-31236156762d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:29.947949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fd21b815-894e-4e2f-918c-31236156762d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:29.971767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911d147-7ce5-43e9-9552-d8b2f77f72c1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.005717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.006516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.006803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.007080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.014131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.016156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.020628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 419/850] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:30.021652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21b815-894e-4e2f-918c-31236156762d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:30.034630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:30.035624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:30.036614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:30.042447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 432/851] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:30.043464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.137018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21b815-894e-4e2f-918c-31236156762d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.142470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.149340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.161596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6d534-8fd7-4d9e-ab75-b15bd99293b3 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.173388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.174061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.174351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.174633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.175414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:30.175731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.177130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.189106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 420/852] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:30.189928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.204834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.205728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.206019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.206300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.207148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:30.207499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.208945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.223473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 433/853] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:30.224295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.303075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00e922-4696-4ea1-ba8b-741b579da006 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.339352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.340273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.340561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.340839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.341496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.341724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67956d9c-8367-4397-bee4-5f3c905877eb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.342880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:30.355060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:30.356140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:30.357003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 421/854] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:30.357806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.385841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.386635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.386917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.387341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.395717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.397159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.398256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.400265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:30.414375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:30.415425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 434/855] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:30.416376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.474123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112f4e-3772-43ff-a18f-e21b68d89507 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.514565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.515265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.515579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.515901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.516691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=c58ac60d4c7441a4ba1053cbbb14b907) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:30.517009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.518462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.524231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76e65f-4ff7-4847-961d-457668c553ce None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.525099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 422/856] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/role_assignments?user.id=c58ac60d4c7441a4ba1053cbbb14b907&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:30.525860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.580775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.582132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.582622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.583323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.584704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:30.585211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.586861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33f350-0d3f-4cc2-a7d8-bc67f2361cf1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.595798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 435/857] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:30.598526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52a2e-316f-4a26-94bb-1c11cc8f1060 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.599164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52a2e-316f-4a26-94bb-1c11cc8f1060 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.599667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52a2e-316f-4a26-94bb-1c11cc8f1060 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.602473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 436/858] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:30.603879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52a2e-316f-4a26-94bb-1c11cc8f1060 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.645400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.655037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817424-e61a-449d-823f-9ea81f055db1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.678853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.695249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.696049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.696570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.696570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.697521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:30.697972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.699351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c3148-5009-4a4d-8fd7-f4cfefd6e1ce None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.719280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 423/859] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:30.720523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.720813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.721090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.730241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52a2e-316f-4a26-94bb-1c11cc8f1060 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.738721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:30.740115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:30.770924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.771915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.772195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.772477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.773192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:30.773499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:30.774880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:30.779782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 437/860] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:30.780581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.821739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 424/861] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:30.846190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:30.905703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1f3d-ce3d-4cd9-8775-cec04fe028fa None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:30.936947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.945996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:30.958017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:30.958017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:30.958017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:30.983817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:30.997333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5190862-24c5-414e-b08f-c1b564650c7c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.028836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.030448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.047727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.048830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.049114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.049393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.050593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.052467 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:31.070911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:31.071936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:31.072780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 425/862] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:31.078438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:31.160183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 438/863] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:08:31.165961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:31.212976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:31.239424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fae622-1902-4ed4-9d3b-1f1effb9a68b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.278722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.279827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.280352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.281440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.282394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:31.282971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.284621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.294933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 426/864] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:31.296643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:31.307566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f82d5-6600-479f-a0a2-ba1eab153ebc None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.319045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:31.355983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.357147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.357433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.357730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.370985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.372650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:31.396825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:31.397815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:31.398733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:31.398733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:31.398733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:31.398733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:31.403179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 439/865] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:31.404566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:31.429484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f93f-f433-4dc3-9f07-fe813a6fc6ba None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.467957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.469566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.470067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.470567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.471661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:31.472257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.473846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.486651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 427/866] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:31.511890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:31.540899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7079a-eb39-4546-8acb-93007a61f3dd None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.581450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.582186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.582475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.582758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.583518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:31.583832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.585234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.598711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 440/867] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:31.604325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:31.703588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:31.726110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5e7c-015a-4e2b-ab6c-324982667d2f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.762833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.763677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.763964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.764246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.771859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.773303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:31.787244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:31.788179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:31.789507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 428/868] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:31.794693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:31.901821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029af12b-0fd4-4656-8c57-ad8fdefb7942 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:31.938010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:31.938684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:31.938971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:31.939250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:31.940070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:31.940384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:31.941838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5653c03-2d59-4912-9d92-0831080af6e7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:31.951502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:31.955223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 441/869] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:31.961323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:32.074500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431db44f-8c8d-4789-8cdc-b0e8abaf76e8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:32.109882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:32.110538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:32.110823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:32.111097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:32.111965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=400527de384349f58866dd9e39468307) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:32.112279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:32.113724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e830a63-2c98-436f-bc50-d27f0afbdaeb None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:32.120381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 429/870] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:31 2022] GET /identity/v3/role_assignments?user.id=400527de384349f58866dd9e39468307&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:32.941928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7ff9e-dd12-4f94-bbf1-58fd2dba8cab None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:32.942416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7ff9e-dd12-4f94-bbf1-58fd2dba8cab None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:32.942416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7ff9e-dd12-4f94-bbf1-58fd2dba8cab None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:32.944021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 442/871] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:32.949496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95bbf4-f3aa-4c13-ad40-345918c329d9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:32.949918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95bbf4-f3aa-4c13-ad40-345918c329d9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:32.950096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95bbf4-f3aa-4c13-ad40-345918c329d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:32.955353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:32.967641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9a95bbf4-f3aa-4c13-ad40-345918c329d9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:32.968856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9a95bbf4-f3aa-4c13-ad40-345918c329d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:33.038078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 430/872] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:33.061599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7ff9e-dd12-4f94-bbf1-58fd2dba8cab None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:33.150940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8397a-0024-4d94-827c-0c3871298b66 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.151265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8397a-0024-4d94-827c-0c3871298b66 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.151527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8397a-0024-4d94-827c-0c3871298b66 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.153138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 431/873] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:33.158855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.159423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.159938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.174480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7ff9e-dd12-4f94-bbf1-58fd2dba8cab None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:33.178382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:33.179801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:33.212359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:33.213097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.213379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.213655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.228177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:33.238679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:33.239673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:33.240541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 443/874] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jun 1 04:08:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:33.246316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:33.267512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 432/875] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:33.287068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:33.321146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:33.362362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:33.370954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b22fde-dbfa-4d57-b89b-8d09803f3289 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:33.410540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:33.411213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.411542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.411822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.412582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:33.412892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:33.414294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:33.416486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc734e3-e1c2-4e77-b5c4-dab9cb3a5198 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:33.424196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 444/876] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:08:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:33.431642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:33.459863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:33.460796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.461176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.461548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.462441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:33.463956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:33.531709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 433/877] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:33 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:33.564258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55149a3f-c1be-46e0-ba0a-75c422f33208 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:33.591015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:33.612295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:33.613102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.613384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.613657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.631359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:33.631359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:33.649196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:33.650184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:33.651324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:33.651324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:33.651324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:33.651324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:33.651324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 445/878] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:33.656639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:33.768759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533843d-00d6-4f66-bb95-a7ddfdebe810 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:33.793910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:33.805904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:33.806576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.806855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.807125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.808118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:33.808238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:33.809566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa81a60-7186-439a-a7f9-8c86f172bfc0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:33.823270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 434/879] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:33.828996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:33.877785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06917a-8b4a-4b73-9d22-fa92adb64c16 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.878441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06917a-8b4a-4b73-9d22-fa92adb64c16 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.878924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06917a-8b4a-4b73-9d22-fa92adb64c16 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.881134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 435/880] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:33.888247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.888803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.889288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:33.906934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:33.908497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:33.947621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07155-e0c0-476b-9de7-99ab442e84a1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:33.986847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 436/881] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:33.994424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:33.995586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:33.996078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:33.996585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.005765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.007271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.007859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:34.027445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:34.028473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:34.030042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 446/882] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:08:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:34.036235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.083564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:34.128372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828792b5-b492-4ebd-b7f5-e0dc50323aa7 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:34.143917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:34.168223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d771e-939e-4c92-9c88-8ec26a327d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:34.168928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d771e-939e-4c92-9c88-8ec26a327d3c None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.169519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d771e-939e-4c92-9c88-8ec26a327d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.169810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d771e-939e-4c92-9c88-8ec26a327d3c None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.171175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d771e-939e-4c92-9c88-8ec26a327d3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.173963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d771e-939e-4c92-9c88-8ec26a327d3c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.178861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27979175-0bc4-4300-a0e7-141106de442e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:34.214907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:34.215640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.215923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.216250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.217084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:34.217396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.219488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.237300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 447/883] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:34.242558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.289565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 437/884] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:34 2022] POST /identity/v3/services => generated 269 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:08:34.357511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:34.373851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a992ce-a570-4609-b0b1-560189c86d72 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:34.423716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:34.424774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.425278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.425763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.464258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060459b-34da-40fc-b1a5-2dc37afad63d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.464590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060459b-34da-40fc-b1a5-2dc37afad63d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.464879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060459b-34da-40fc-b1a5-2dc37afad63d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.466532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 438/885] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:34.472718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.473026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.473289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.492035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:34.493246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:34.518682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.520237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.583227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 439/886] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:34.607475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.617652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:34.637975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 448/887] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 01 04:08:34.644282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.737747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d005df4-9dbd-47db-b2a5-2872a4075a42 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:34.739485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:34.779732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9c93-6556-4229-9f88-ddf9272fcf66 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:34.784637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:34.785518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.785806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.786087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.786752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.788176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:34.804885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:34.805922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:34.806762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:34.811210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 440/888] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:34.813587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.829586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:34.831138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.831723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.832207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.843791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.845718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:34.865675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:34.866646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:34.867594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:34.867594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:34.867594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:34.867594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:34.871633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 449/889] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:34.882531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:34.928613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:34.936468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5818a-1cc7-4da8-b705-20de0b2dee83 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:34.974074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:34.974988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:34.975456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:34.975884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:34.976872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:34.977344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:34.979007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:34.985382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 441/890] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:34.990820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:35.005347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc119ef-e262-4bb8-a5a9-8a1133c761d4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:35.032948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:35.053023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:35.053696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.053985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.054288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.055540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:35.055969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:35.057513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:35.075355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 450/891] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:35.080004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:35.125607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75b17-898d-4627-9dfc-3701934adf79 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:35.169816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:35.170632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.170990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.171892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.172954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:35.173333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:35.175775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:35.183462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 442/892] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:35.211441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:35.218622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ccb5-f45a-4b80-b98d-0e6f6f64987a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:35.258620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:35.259847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.260335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.260796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.269980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:35.271921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:35.288884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:35.289857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:35.290783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:35.293130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 451/893] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:35.300496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:35.413323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:35.422089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600f816-b251-4fc5-8822-00d369c60e47 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:35.461392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:35.462067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.462356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.462635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.463466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:35.463784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:35.465229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf321d5a-3053-4060-88ba-a4369758c8c1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:35.478507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 443/894] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:35.484457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:35.598095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31ac3e-94d7-4b5c-abbf-aa5586446565 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:35.646767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:35.647474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.647848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.648186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.649022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] RBAC: Adding query filter params (role.id=8290d13fe713458f8443fa31d743e3bc, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=c58ac60d4c7441a4ba1053cbbb14b907) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:35.649336 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:35.650770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb964e38-532a-4f65-a386-a8e80f5a06b0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:35.657784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 452/895] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:35 2022] GET /identity/v3/role_assignments?user.id=c58ac60d4c7441a4ba1053cbbb14b907&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=8290d13fe713458f8443fa31d743e3bc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:35.925468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905bb0b-502c-4a88-b17a-09bf270f16e0 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.925870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905bb0b-502c-4a88-b17a-09bf270f16e0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.926100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905bb0b-502c-4a88-b17a-09bf270f16e0 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.927776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 444/896] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:35.933208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:35.933506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:35.933791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:35.952160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:35.953904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:35.969840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:36.026114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 453/897] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:36.050097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905bb0b-502c-4a88-b17a-09bf270f16e0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:36.162093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905bb0b-502c-4a88-b17a-09bf270f16e0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:36.198496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:36.199225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:36.199543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:36.199827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:36.200486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:36.201848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:36.215233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:36.216253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:36.217070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 445/898] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:36.222745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:36.333471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e37a2-4f09-4b38-be92-7f4ff0877e75 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:36.369058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:36.369723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:36.370005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:36.370281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:36.370994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:36.371321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:36.372697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:36.378755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 454/899] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:36.384907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:36.407018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:36.499953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbae63-31ca-4108-a224-94465bd3aff8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:36.532068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:36.539329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:36.539329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:36.539329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:36.539329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:36.539329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:36.539329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb63e16-5f6f-4aae-bf15-fd02c5c93235 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:36.541241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 446/900] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:36.555326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:36.565137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:36.665704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8364a-fecf-4119-b36d-59fb8acb18d1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:36.684630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bafb5-f770-4525-8701-f9fcc645719d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:36.684914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bafb5-f770-4525-8701-f9fcc645719d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:36.685358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bafb5-f770-4525-8701-f9fcc645719d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:36.687433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 447/901] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:36.692234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:36.692676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:36.693098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:36.721040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:36.723502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:36.737421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:36.738361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:36.738798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:36.739284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:36.740254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] RBAC: Adding query filter params (interface=public, service_id=9067d1bdd61140ed90c331f597f2250b, region_id=RegionOne) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:36.740734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:36.742313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:36.759866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 455/902] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:36 2022] GET /identity/v3/endpoints?service_id=9067d1bdd61140ed90c331f597f2250b&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:36.760975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:36.871254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 448/903] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:36.888159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:36.907641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:36.943698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:36.983738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb449e3b-3296-41cf-b692-247f8826ad29 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.050572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.052066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.052548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.053087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.053972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.055032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87654-7060-4c85-a2f4-8625a728b43b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.056254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.071755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 456/904] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:36 2022] GET /identity/v3/services => generated 1268 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:08:37.094329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.095167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.095500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.095801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.096442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.097852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:37.112649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:37.113678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:37.114535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 449/905] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Jun 1 04:08:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:37.122208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:37.243241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e080c6d-59e8-4f34-8625-82426a82b35e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.284884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.285557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.285845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.286126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.286838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:37.287150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.288564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.293695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 457/906] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:37.299573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:37.389158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:37.413932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9728a1-0801-400e-a257-fec629fe9ccc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.449240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.449915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.450204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.450486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.451201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:37.451539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.453453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0646-1f9b-4e6e-b290-762801cc8cb9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.459079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 450/907] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:37.468979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:37.478634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:37.575201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eff0b0-a900-40a4-81c4-f0fbc6129965 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.586380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5c85c-5f7b-4415-9640-1a12d8280b5d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.586713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5c85c-5f7b-4415-9640-1a12d8280b5d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.586941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5c85c-5f7b-4415-9640-1a12d8280b5d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.588952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 451/908] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:37.598059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.598468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.598846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.617660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.617871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:37.623889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:37.627531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.628000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.628438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.630243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] RBAC: Adding query filter params (interface=public, service_id=a66ade556de048f9862333816fac9c78, region_id=RegionOne) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:37.630711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.632416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.638649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 458/909] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:37 2022] GET /identity/v3/endpoints?service_id=a66ade556de048f9862333816fac9c78&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:37.645311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:37.711326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 452/910] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:37.726555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:37.735131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:37.769534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8068c7-8ac6-4bb6-a413-7cedbbd61eeb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.807172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2dc19-ed74-4493-9757-da96ea062559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.808196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2dc19-ed74-4493-9757-da96ea062559 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.808740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2dc19-ed74-4493-9757-da96ea062559 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.809267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2dc19-ed74-4493-9757-da96ea062559 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.810225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2dc19-ed74-4493-9757-da96ea062559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.812406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2dc19-ed74-4493-9757-da96ea062559 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.813065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:37.825059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 459/911] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:37 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:08:37.848109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096634-0f57-4f16-adba-09d7d61238de None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:37.886203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404ac3f-9fc2-420e-8fda-52bab2abb239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:37.886897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ac3f-9fc2-420e-8fda-52bab2abb239 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.887185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ac3f-9fc2-420e-8fda-52bab2abb239 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.887500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ac3f-9fc2-420e-8fda-52bab2abb239 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:37.888259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404ac3f-9fc2-420e-8fda-52bab2abb239 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:37.889682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404ac3f-9fc2-420e-8fda-52bab2abb239 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:37.971866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 453/912] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:37 2022] POST /identity/v3/services => generated 241 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:08:37.998289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2417-e4af-4f72-9889-9a08e4c254a8 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:37.998610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2417-e4af-4f72-9889-9a08e4c254a8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:37.998853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a2417-e4af-4f72-9889-9a08e4c254a8 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:38.001427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 460/913] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:38.007901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:38.008211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:38.008471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:38.027815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:38.029108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:38.115470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 454/914] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:38.139242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2417-e4af-4f72-9889-9a08e4c254a8 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:38.253191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a2417-e4af-4f72-9889-9a08e4c254a8 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:38.292376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:38.293299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:38.293586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:38.293862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:38.296829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:38.306954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:38.315387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:38.316303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:38.317542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 461/915] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jun 1 04:08:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:38.322879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:38.387003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:38.433389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb101b-d034-4828-be6f-c84dd886af66 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:38.468494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:38.469190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:38.469479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:38.469759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:38.470523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:38.470831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:38.472281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:38.481083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 455/916] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:08:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:38.487262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:38.594387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2982cc7-e704-44e5-8986-154dac5c955e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:38.646264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:38.647067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:38.647371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:38.647653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:38.659291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:38.660834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:38.678622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:38.679581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:38.680899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:38.680899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:38.680899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:38.680899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:38.680899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 462/917] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:38.685572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:38.793937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184c57-f32a-4a16-b528-99d383a0877b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:38.820189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:38.829547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:38.830356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:38.830648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:38.830933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:38.831826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:38.832137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:38.833524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:38.846613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 456/918] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:38.851819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:38.901575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:38.961473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938802ed-3e94-4bd2-9f03-8b70ffea711d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:38.995668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:38.996486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:38.996768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:38.997042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.005321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.006760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:39.022048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:39.023132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:39.024116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 463/919] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:08:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:39.029190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.135044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b71f-2889-43ed-b1b3-5cb64a1c10cd None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:39.177684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:39.178386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.178678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.178964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.179889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:39.180207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.181696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.203080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 457/920] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:39.208548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.315153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726d857-2ab4-490c-827a-7cf5a7223be8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:39.322335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:39.349666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:39.350464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.350753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.351027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.363278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.364770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c6f9fdd4-ab7e-4493-939c-e35765a95863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:39.382916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:39.383902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:39.384780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:39.384780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:39.384780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:39.384780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:39.384780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 464/921] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:39.392501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.428979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:39.501381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab0d70-9676-4e85-95f4-48b8e1bb9d11 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:39.537805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:39.538492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.538805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.539194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.540830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e4506-a073-4ad5-9464-f479db2e96d0 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.544892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e4506-a073-4ad5-9464-f479db2e96d0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.544892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e4506-a073-4ad5-9464-f479db2e96d0 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.545094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:39.545094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.545094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.545502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 465/922] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:39.548870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.549162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.549446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.559464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 458/923] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:39.568272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.569122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:39.570328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:39.655461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 466/924] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:39.677516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.682806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:39.689382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce0020-7c07-4f25-bd5e-1b006b24c1df None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:39.725605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:39.727037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.727476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.727832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.735908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.737474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:39.753335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:39.754524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:39.755468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:39.760101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 459/925] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:39.761428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.779425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:39.793390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069435a5-c5ca-48af-b522-509ba9294ca9 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:39.837240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:39.838226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.838617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.839095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.840002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.841546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:39.856027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:39.857096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 01 04:08:39.857929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:39.859888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 467/926] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:39.867519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.876726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64646ab-ad04-49c8-8d02-d7f104b4d0a7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:39.913713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:39.914730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:39.915084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:39.915484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:39.916360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:39.916741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:39.918254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:39.937607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 460/927] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:39.938610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:39.957902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:39.985963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef06690-865d-40b3-83b1-fd21f1ed7a39 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.026777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.027887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.028389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.028862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.029781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:40.030290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.032488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.038498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 468/928] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:40.044746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.062991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1a3a5-2225-45a9-9b5c-eb5a9cfc499e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.067142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:40.107826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa28e7-e9bc-4cd0-a0e6-595d47b193ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.109302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa28e7-e9bc-4cd0-a0e6-595d47b193ca None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.109642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa28e7-e9bc-4cd0-a0e6-595d47b193ca None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.109871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa28e7-e9bc-4cd0-a0e6-595d47b193ca None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/c58ac60d4c7441a4ba1053cbbb14b907/roles/8290d13fe713458f8443fa31d743e3bc` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.112318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa28e7-e9bc-4cd0-a0e6-595d47b193ca None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=c58ac60d4c7441a4ba1053cbbb14b907, role_id=8290d13fe713458f8443fa31d743e3bc)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.113835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa28e7-e9bc-4cd0-a0e6-595d47b193ca None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.175719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fb51b-a8a5-4689-bb81-c704d5a37c7c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.212976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 461/929] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:08:39 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/c58ac60d4c7441a4ba1053cbbb14b907/roles/8290d13fe713458f8443fa31d743e3bc => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:08:40.220018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.220018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.220018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.220018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.221283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:40.221283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.221283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.223474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c6d3f-9df1-4621-b4fb-88be4e88e0f9 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.223914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c6d3f-9df1-4621-b4fb-88be4e88e0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.224316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c6d3f-9df1-4621-b4fb-88be4e88e0f9 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.225319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 469/930] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jun 1 04:08:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:40.231353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 462/931] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:40.232456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.236406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.236690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.236981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.256551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:40.257896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:40.363333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 463/932] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:40.365802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:40.388336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.393795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b5e4a-586c-4bdd-89b0-e9461c38b20b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.400878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:40.445289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d334e-08f5-4b54-b48d-3e4897be188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.446725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d334e-08f5-4b54-b48d-3e4897be188e None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.447184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d334e-08f5-4b54-b48d-3e4897be188e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.447517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d334e-08f5-4b54-b48d-3e4897be188e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.448266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d334e-08f5-4b54-b48d-3e4897be188e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.449974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d334e-08f5-4b54-b48d-3e4897be188e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.511141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cf5fe-83ab-4825-8020-1177d61b3e1c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.532045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 470/933] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:40.533745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634fe03-831d-488e-8486-3938530bcf7e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.534200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634fe03-831d-488e-8486-3938530bcf7e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.534792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634fe03-831d-488e-8486-3938530bcf7e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.536815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 471/934] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:40.545444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.546338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.546684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.559196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.560613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.561079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.561524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.562999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.564664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.571222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:40.572856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:40.579157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:40.580251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:40.581077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 464/935] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Jun 1 04:08:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:40.587187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.669844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 472/936] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:40.683268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:40.691493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.729756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14499a36-9da2-463e-8e6c-a1cd094f00f3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.767699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:40.778409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.779118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.780648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.780648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.780648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:40.780875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.782178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.789666 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 465/937] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:40.795696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.818893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f4ef1-ddd0-425e-bcea-b09fb61278b5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.864237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.865699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.866381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.867204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.868323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.870376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:40.898812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:40.899948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:40.900799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 473/938] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:40.909796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:40.932303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2f39-417b-4d6d-88f9-89fdf1bd79c8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:40.971409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:40.978661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:40.979724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:40.985199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:40.985199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:40.985199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:40.985199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:40.985199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:40.991909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 466/939] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:40.997012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:41.032483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15682d8-1eaf-4354-a627-3c560fed5108 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:41.072442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:41.079428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:41.080448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:41.080887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:41.081317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:41.082282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:41.082738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:41.084432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:41.095066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 474/940] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:41.096358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:41.133198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ead91e-73e4-44cf-a29f-3cc93906326a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:41.180820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:41.181523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:41.181811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:41.182086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:41.183696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:41.185086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:41.231002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a222c-55f8-469d-96bc-7051b0b8e4f2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:41.274191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:41.275198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:41.275654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:41.276127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:41.277008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:41.277460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:41.279152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:41.285615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 475/941] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:41.291594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:41.318181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 467/942] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:41.382411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:41.440144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3380eaa-2420-486d-ac24-1842fd93a9f6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:41.490270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:41.490947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:41.491230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:41.491542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:41.492343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] RBAC: Adding query filter params (interface=public, service_id=448fd0b5e6b14cf39629fb4089745f6c, region_id=RegionOne) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:41.492657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:41.494086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc75c-a818-44db-a31b-fc74960328d8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:41.500660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 476/943] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:41 2022] GET /identity/v3/endpoints?service_id=448fd0b5e6b14cf39629fb4089745f6c&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:41.506576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:41.566107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:41.630633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b0065-aa25-4d9a-9a79-794b5ea39199 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:41.665089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b1998-a009-42ec-b41b-e76dd40eb244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:41.665762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b1998-a009-42ec-b41b-e76dd40eb244 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:41.666044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b1998-a009-42ec-b41b-e76dd40eb244 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:41.666321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b1998-a009-42ec-b41b-e76dd40eb244 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:41.667007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b1998-a009-42ec-b41b-e76dd40eb244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:41.668520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b1998-a009-42ec-b41b-e76dd40eb244 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:41.676054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 468/944] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:41 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:08:42.590886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd93ffc-0046-4eb1-90c5-9ff4aae32ae6 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:42.591427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd93ffc-0046-4eb1-90c5-9ff4aae32ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:42.591507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd93ffc-0046-4eb1-90c5-9ff4aae32ae6 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:42.593095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 477/945] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:42.598435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:42.598828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:42.598996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:42.616537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:42.617738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:42.695233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 469/946] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:42.718305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd93ffc-0046-4eb1-90c5-9ff4aae32ae6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:42.823486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd93ffc-0046-4eb1-90c5-9ff4aae32ae6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:42.840803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:42.858310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa5e58-3042-4177-a7b8-697dbab43914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:42.859788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa5e58-3042-4177-a7b8-697dbab43914 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:42.860073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa5e58-3042-4177-a7b8-697dbab43914 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:42.860345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa5e58-3042-4177-a7b8-697dbab43914 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:42.873239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa5e58-3042-4177-a7b8-697dbab43914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-77fa5e58-3042-4177-a7b8-697dbab43914 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:42.884140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:42.885255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:42.886305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 478/947] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jun 1 04:08:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:42.888804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:42.936695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:42.988028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ad708-ba81-479d-bc56-35090d4ccd1e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:42.988452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ad708-ba81-479d-bc56-35090d4ccd1e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:42.988832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ad708-ba81-479d-bc56-35090d4ccd1e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:42.990544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 479/948] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:42.995213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a83c6-ce36-4410-a3f7-19a05933facb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:42.995571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:42.995858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:42.996136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.015647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:43.016828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:43.032891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.033930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.034354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.034904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.035689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:43.036000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.037388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.047237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 470/949] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:08:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:43.052487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.111445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 480/950] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:43.133853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.178246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfc1c1-6aa3-4fd5-8c46-1bc15222450f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.185839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:43.215202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.216197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.216557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.217169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.229503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.231104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.246761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c34ac4-1e9f-45bd-b1ce-4b238f7ac402 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:43.250060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:43.251046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:43.252102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:43.252102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:43.252102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:43.252102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:43.252513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 471/951] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:43.260183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:43.261292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.292875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.294463 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.294898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.295337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.296491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.298753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.302081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING keystone.server.flask.common [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 01 04:08:43.355763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.sql.core [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 01 04:08:43.355763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 01 04:08:43.355763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [parameters: {'id': '8c0a107207564adea4ca52162df7a822', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 01 04:08:43.355763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80710) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c0a107207564adea4ca52162df7a822.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c0a107207564adea4ca52162df7a822. Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result = self._query(query) Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application conn.query(q) Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 01 04:08:43.366691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result.read() Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 01 04:08:43.367628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application next(self.gen) Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application yield resource Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application next(self.gen) Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.session.rollback() Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.force_reraise() Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise self.value Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application session.commit() Jun 01 04:08:43.368581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._prepare_impl() Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.session.flush() Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._flush(objects) Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application compat.raise_( Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application flush_context.execute() Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 01 04:08:43.369462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rec.execute(self) Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 01 04:08:43.370493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result = self._query(query) Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application conn.query(q) Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application result.read() Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 01 04:08:43.371319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [parameters: {'id': '8c0a107207564adea4ca52162df7a822', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.372209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:43.373255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c0a107207564adea4ca52162df7a822. Jun 01 04:08:43.374422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.376534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 481/952] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Jun 1 04:08:43 2022] POST /identity/v3/projects => generated 262 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:43.382929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.385692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198747bd-8d8a-4cef-8103-5a2108452d28 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.439280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.440251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.440699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.441120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.441995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:43.442963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.444865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.467130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 472/953] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:43.469459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.497913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:43.498121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8824a-fa94-4c01-9f58-b0323534d815 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.538496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.539423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.539875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.540252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.549094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.551729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:43.573095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:43.574256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:43.575568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:43.579044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 482/954] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:43.586514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.612644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311f866-7d2b-474c-83dd-2191cb9bc0a9 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.626017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:43.661285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.662282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.662675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.663040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.673784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.675530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:43.698185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:43.699450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 01 04:08:43.700653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:43.704773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 473/955] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:43.712386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.729749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7d9a-8350-4311-92e9-a762eba2631e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.768859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.769720 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.770123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.770502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.771444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:43.771868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.773518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5836f3-45d0-463d-8e70-959512f54efe None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.797176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 483/956] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:43.805126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:43.829228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec3c7b-73b0-4a81-bfc6-420965862d1f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:43.853933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90398ee-9269-43dd-9a81-0bdc64953336 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.854276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90398ee-9269-43dd-9a81-0bdc64953336 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.854503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90398ee-9269-43dd-9a81-0bdc64953336 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.856979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 484/957] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:43.863246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.863590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.863871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.871023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:43.871770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:43.872058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:43.872334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:43.873219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:43.873527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:43.875620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:43.884605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:43.886308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:43.898233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 474/958] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:43.904267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:43.971465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 485/959] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:43.992406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.010790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.033687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a316071-a182-498a-8ff5-06349d8a3823 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.077906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480b5e-2757-42e3-ae17-7a0d48f7e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.078562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480b5e-2757-42e3-ae17-7a0d48f7e655 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.078847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480b5e-2757-42e3-ae17-7a0d48f7e655 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.079125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480b5e-2757-42e3-ae17-7a0d48f7e655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.083232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.114397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5351e1e-47d4-4cab-be20-7501ba097507 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.154634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.155404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.155692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.155970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.156078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d480b5e-2757-42e3-ae17-7a0d48f7e655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.156662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.157499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d480b5e-2757-42e3-ae17-7a0d48f7e655 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.158024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-999cb6a4-45fa-4bc8-9251-283a0a4e95e7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:44.170349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:44.171988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.172965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 486/960] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:44.173794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123ef5-591c-483e-95e8-d7937303d548 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.174539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123ef5-591c-483e-95e8-d7937303d548 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.174539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123ef5-591c-483e-95e8-d7937303d548 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.179806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 487/961] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:44.179806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2123ef5-591c-483e-95e8-d7937303d548 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.269205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 475/962] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:08:44.270610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.270904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.271184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.273721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.292063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:44.293699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:44.299107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2123ef5-591c-483e-95e8-d7937303d548 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.329502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.336038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.336572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.336922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.337206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.337917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] RBAC: Adding query filter params (name=compute) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:44.338240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.339627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.350467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 488/963] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:44.350467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.376890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 476/964] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:44.377578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.461312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af954-0c4e-42af-a430-0eeb1d4057ce None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.486763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b814b-b770-4a75-b1c2-6bce4eb552a4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.496376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.497179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.497465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.497744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.508559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.510047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.511867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.526984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.527936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.528403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.528954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:44.529107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:44.530128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:44.531646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:44.531646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:44.531646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:44.531646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.531646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 489/965] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:44.531944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:44.531944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.533328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.536070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.547329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 477/966] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:44.547329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.590296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.665345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab5ffe-4ce6-4d5a-baab-177e2a8f9462 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.685626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc165c1f-af5f-4fae-b08e-a71ca0ddb3e5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.707097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.709205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.709783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.710279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.711209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.712980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:44.726821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:44.728247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:44.729141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:44.731697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 490/967] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:44.734772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.737767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.738714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.738998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.739272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.740955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:44.741285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.742804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.758249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 478/968] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:44.764396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.825273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.859958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:44.869747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc5ff7-4d54-4b25-8f92-df8072c5dc67 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.871640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a85b86-5ab6-42b0-aa89-918ec88504b0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:44.906821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.907657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.907947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.908227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.910512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:44.911503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:44.911999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:44.912481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:44.913493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:44.914173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.915039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:44.916057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.916497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:44.926602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 491/969] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:44.927743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:44.930981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:44.931897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:44.932774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 479/970] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:44.937980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:45.044118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a661f-2cb9-463a-80ad-dca7ae35a673 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:45.053141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31004c8e-b525-4794-85e9-c5f8175d4ac2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:45.074975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:45.093760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:45.094884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:45.095491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:45.095970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:45.096959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:45.097470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:45.099074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:45.099950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:45.100612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:45.101049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:45.101463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:45.102123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:45.102430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:45.103902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:45.106624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 492/971] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:45.114018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:45.125450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 480/972] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:45.132049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:45.145342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:45.231012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0457f6-902c-4c9c-aa16-9e510a18ed1c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:45.266349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ef446-41a0-4c6c-8c6e-738bff6e5635 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:45.273626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a04a96-6356-4625-bb2f-56fa5275b64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:45.274297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a04a96-6356-4625-bb2f-56fa5275b64d None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:45.274579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a04a96-6356-4625-bb2f-56fa5275b64d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:45.274859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a04a96-6356-4625-bb2f-56fa5275b64d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:45.275644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a04a96-6356-4625-bb2f-56fa5275b64d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:45.277504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a04a96-6356-4625-bb2f-56fa5275b64d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:45.310020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:45.310020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:45.310020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:45.310020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:45.310393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] RBAC: Adding query filter params (role.id=8290d13fe713458f8443fa31d743e3bc, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=c58ac60d4c7441a4ba1053cbbb14b907) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:45.310603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:45.312781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993f824-39b3-4eca-af28-7042501377b0 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:45.319197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 481/973] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:45 2022] GET /identity/v3/role_assignments?user.id=c58ac60d4c7441a4ba1053cbbb14b907&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=8290d13fe713458f8443fa31d743e3bc => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:45.368817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 493/974] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:08:46.201929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ff092-9aa9-46e5-9022-8c62f6a2c0c5 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.202978 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ff092-9aa9-46e5-9022-8c62f6a2c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.203086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ff092-9aa9-46e5-9022-8c62f6a2c0c5 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.204738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 482/975] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:46.210258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.211082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.211617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.229422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:46.230940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:46.269237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:46.308399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 494/976] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:46.333403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ff092-9aa9-46e5-9022-8c62f6a2c0c5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:46.417763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:46.444303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ff092-9aa9-46e5-9022-8c62f6a2c0c5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:46.480678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:46.481289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.481575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.481852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.482590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:46.484226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:46.485597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING keystone.server.flask.common [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 01 04:08:46.538413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.sql.core [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 01 04:08:46.538413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 01 04:08:46.538413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [parameters: {'user_id': 'b8899e7cf48f41e59576bba174aa04ae', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 01 04:08:46.538413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80711) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application result = self._query(query) Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application conn.query(q) Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application result.read() Jun 01 04:08:46.555750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 01 04:08:46.556731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application next(self.gen) Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application yield resource Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application next(self.gen) Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self.session.rollback() Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self.force_reraise() Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise self.value Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application session.commit() Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 01 04:08:46.557767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._prepare_impl() Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self.session.flush() Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._flush(objects) Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application compat.raise_( Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application flush_context.execute() Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rec.execute(self) Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 01 04:08:46.559392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 01 04:08:46.560299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application result = self._query(query) Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application conn.query(q) Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application result.read() Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 01 04:08:46.561161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b8899e7cf48f41e59576bba174aa04ae', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:46.562564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 01 04:08:46.566079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:46.568214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 483/977] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jun 1 04:08:46 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:46.569089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:46.673460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba2b79-50ea-4769-ad23-513b8717802d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:46.709254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:46.710338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.710622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.710900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.718572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:46.720041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-34e7e5b3-da41-4fee-9215-c84825cc4afb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:46.734777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:46.735776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 01 04:08:46.736721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:46.740695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 495/978] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:08:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:46.742348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:46.853969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61ca74-75b8-4045-bddc-72fdd13ceeb2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:46.888289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:46.888960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.889242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.889559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.890878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:46.891184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:46.892839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e6976-b14d-4934-b2f9-e1dc2780ac0f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:46.902912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:46.931415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 484/979] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jun 1 04:08:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:46.959508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807506f-7c78-4ff0-b723-0cb1c6a52627 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.959907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807506f-7c78-4ff0-b723-0cb1c6a52627 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.960265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807506f-7c78-4ff0-b723-0cb1c6a52627 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.962113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 496/980] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:46.968136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a52551-38d1-49fe-a6a8-924bf5e21980 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:46.968434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a52551-38d1-49fe-a6a8-924bf5e21980 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:46.968726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a52551-38d1-49fe-a6a8-924bf5e21980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:46.988512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-02a52551-38d1-49fe-a6a8-924bf5e21980 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:46.990228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-02a52551-38d1-49fe-a6a8-924bf5e21980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:47.089112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 485/981] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:47.116098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807506f-7c78-4ff0-b723-0cb1c6a52627 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:47.150637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:47.255764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807506f-7c78-4ff0-b723-0cb1c6a52627 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:47.302452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e13da-7bea-498e-b87d-e2977ea701fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:47.303131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e13da-7bea-498e-b87d-e2977ea701fe None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:47.303449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e13da-7bea-498e-b87d-e2977ea701fe None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:47.303727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e13da-7bea-498e-b87d-e2977ea701fe None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:47.304664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e13da-7bea-498e-b87d-e2977ea701fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:47.306073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e13da-7bea-498e-b87d-e2977ea701fe None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:47.400082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 497/982] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:47 2022] POST /identity/v3/services => generated 242 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:47.758633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f075e7d-f82a-4423-8c3c-8abecbecbb56 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:47.759183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f075e7d-f82a-4423-8c3c-8abecbecbb56 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:47.759183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f075e7d-f82a-4423-8c3c-8abecbecbb56 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:47.760652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 486/983] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:47.766387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:47.766736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:47.767026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:47.781803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a4adc-e4f3-477b-92e2-19643d78198d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:47.782108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a4adc-e4f3-477b-92e2-19643d78198d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:47.782376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a4adc-e4f3-477b-92e2-19643d78198d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:47.783816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 487/984] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:47.785619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:47.786821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:47.788913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:47.789206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:47.789490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:47.806404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:47.807623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:47.872833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 498/985] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:47.881648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 488/986] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:47.895614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:47.905097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:47.912585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:47.968783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:48.013524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51f508-8f31-4260-940b-901df73d58a4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.024556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a67b0-9a98-4803-bd9f-c10af134028f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.050899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.051923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.052106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.052361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.053020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.054408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.061961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.062779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.063063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.063364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.064024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.065385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:48.066917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:48.068006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:48.068877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 499/987] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Jun 1 04:08:47 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:48.076383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:48.082195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:48.083224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:48.084109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:48.087647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 489/988] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:48.089320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:48.169915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:48.192281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faac38-f0f3-40f9-9ba1-8a702e76fae8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.215781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bef61-4ddf-4c96-9c91-47a8b73b5838 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.229599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.230267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.230544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.230820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.231548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] RBAC: Adding query filter params (name=image) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:48.231854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.233260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.239661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:48.239890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 500/989] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:48.245493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:48.262435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.263237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.263715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.264000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.265412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:48.265722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.267125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.274406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 490/990] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:48.279604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:48.359248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdd4b6-b519-469d-b336-918ec024f253 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.396230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.396901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.397190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.397469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.398177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] RBAC: Adding query filter params (type=image) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:48.398488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.398643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d522d-ee8a-41cf-8d51-8b3f2ca081af None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.399923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.415357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 501/991] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:48.448657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.449260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.449545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.449821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.450548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:48.450855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.452452 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.458323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 491/992] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:48.464374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:48.474356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:48.582638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429d37f-733f-4142-b6c0-63e18e22705c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.629697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.630622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.631143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.631716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.632858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] RBAC: Adding query filter params (interface=admin, service_id=448fd0b5e6b14cf39629fb4089745f6c, region_id=RegionOne) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:48.633087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.634652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0e040-c889-4098-bef4-d2275241c967 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.640722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 502/993] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Jun 1 04:08:48 2022] GET /identity/v3/endpoints?service_id=448fd0b5e6b14cf39629fb4089745f6c&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:48.647602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:48.708061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:48.766503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b385c5b-b7f0-487e-ae13-2b64d8c0976b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:48.802630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3919611-f12d-4596-8c26-70f301ef206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:48.803329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919611-f12d-4596-8c26-70f301ef206a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:48.803632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919611-f12d-4596-8c26-70f301ef206a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:48.803912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3919611-f12d-4596-8c26-70f301ef206a None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:48.804601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3919611-f12d-4596-8c26-70f301ef206a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:48.805987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3919611-f12d-4596-8c26-70f301ef206a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:48.819328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 492/994] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:48 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:08:49.706864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4861740-9998-48b4-9440-a310b1d95be6 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:49.707562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4861740-9998-48b4-9440-a310b1d95be6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:49.707562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4861740-9998-48b4-9440-a310b1d95be6 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:49.708853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 503/995] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:49.714179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:49.714548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:49.714837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:49.732206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:49.733993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:49.839369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 493/996] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:49.862833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4861740-9998-48b4-9440-a310b1d95be6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:49.958371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:50.007930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4861740-9998-48b4-9440-a310b1d95be6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:50.052991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:50.054058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.054491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.054942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.055831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:50.057356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:50.069891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:50.071069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:50.071935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:50.075483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 504/997] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:50.078577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:50.135578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:50.199292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd42fc-b928-44ab-982d-d8851a262ab6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:50.245131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:50.245725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.246012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.246291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.247017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:50.247349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:50.248747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:50.259992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 494/998] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:50.261297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:50.370853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31739279-537c-4ef1-911a-3779f23a1609 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:50.409961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:50.410633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.410915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.411188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.411923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] RBAC: Adding query filter params (type=compute) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:50.412228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:50.413740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:50.415065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:50.419947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 505/999] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:50.426042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:50.531477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195747c0-a05c-42f5-9731-16b9c4eba49e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:50.533946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:50.572417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:50.573090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.573401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.573657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.574532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] RBAC: Adding query filter params (interface=public, service_id=38878bad8bc24b53bbb0eba9e6bf1d7e, region_id=RegionOne) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:50.574848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:50.576269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d68f1a-e9d6-4885-be8a-3f0c83bf4743 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:50.586810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 495/1000] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:50 2022] GET /identity/v3/endpoints?service_id=38878bad8bc24b53bbb0eba9e6bf1d7e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:50.587989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:50.704302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac038-2f43-4936-b390-348b5906f7ba None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:50.713299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4253a5-743a-41f2-b0b2-f59c3b4058e2 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.713585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4253a5-743a-41f2-b0b2-f59c3b4058e2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.713872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4253a5-743a-41f2-b0b2-f59c3b4058e2 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.715424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 496/1001] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:50.721169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0cca8-556b-4ae1-a49b-7494889f7ef4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.721461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0cca8-556b-4ae1-a49b-7494889f7ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.721738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0cca8-556b-4ae1-a49b-7494889f7ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.738714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4ee0cca8-556b-4ae1-a49b-7494889f7ef4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:50.739898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4ee0cca8-556b-4ae1-a49b-7494889f7ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:50.742075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:50.742750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:50.743033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:50.743327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:50.744012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:50.745393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:50.759520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 506/1002] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:50 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:08:50.831441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 497/1003] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:50.854781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:50.925052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:50.974400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102df90-4fe4-4a2c-998d-b32c5b25d3f2 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:51.013034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb6a36-e241-4dfd-9024-70bed70aa78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:51.013702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb6a36-e241-4dfd-9024-70bed70aa78b None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.013987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb6a36-e241-4dfd-9024-70bed70aa78b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.014260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb6a36-e241-4dfd-9024-70bed70aa78b None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.015566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb6a36-e241-4dfd-9024-70bed70aa78b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:51.016937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb6a36-e241-4dfd-9024-70bed70aa78b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:51.116299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 507/1004] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:50 2022] POST /identity/v3/services => generated 242 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:51.137378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db95c-1b5b-4ea0-945c-7fd04b2f509a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.138135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db95c-1b5b-4ea0-945c-7fd04b2f509a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.138135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58db95c-1b5b-4ea0-945c-7fd04b2f509a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.140205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 498/1005] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:51.146138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533021c0-be78-499d-80ce-6d99db47466f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.146448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533021c0-be78-499d-80ce-6d99db47466f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.146721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533021c0-be78-499d-80ce-6d99db47466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.165370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-533021c0-be78-499d-80ce-6d99db47466f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:51.166601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-533021c0-be78-499d-80ce-6d99db47466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:51.255354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 508/1006] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:51 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:51.279235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58db95c-1b5b-4ea0-945c-7fd04b2f509a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:51.409383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:51.415230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58db95c-1b5b-4ea0-945c-7fd04b2f509a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:51.454923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:51.455781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.456070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.456346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.457010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:51.458406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:51.471411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:51.472530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:51.473392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 499/1007] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jun 1 04:08:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:51.478653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533021c0-be78-499d-80ce-6d99db47466f None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:51.535001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:51.590366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533021c0-be78-499d-80ce-6d99db47466f None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:51.637464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:51.638142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.638548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.638828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.639612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:51.639924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:51.641303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:51.651141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 509/1008] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:51.652775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:51.762213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2bbb4-51ee-46ba-b3ca-f5957412acd6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:51.801683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:51.802360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.802651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.802934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.803679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:51.803992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:51.805428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262f978-2f59-483e-bbd9-44821e49652e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:51.814986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 500/1009] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jun 1 04:08:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:51.816685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:51.927127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796d252-5bea-42b9-9e98-7286e40bd6fa None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:51.942708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:51.971159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef232f-bbc0-433c-bb5a-91a23bbba96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:51.971919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef232f-bbc0-433c-bb5a-91a23bbba96d None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:51.972202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef232f-bbc0-433c-bb5a-91a23bbba96d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:51.972478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef232f-bbc0-433c-bb5a-91a23bbba96d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:51.973248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef232f-bbc0-433c-bb5a-91a23bbba96d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:51.974989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef232f-bbc0-433c-bb5a-91a23bbba96d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:52.057061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 510/1010] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:53.042688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdc869-e450-4852-b646-baed10cdbb62 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.043208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdc869-e450-4852-b646-baed10cdbb62 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.043339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdc869-e450-4852-b646-baed10cdbb62 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.044833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 501/1011] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:53.052283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.052283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.053426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.080427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:53.082454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:53.215467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 511/1012] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:53 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:53.235546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdc869-e450-4852-b646-baed10cdbb62 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:53.268058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:53.366630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdc869-e450-4852-b646-baed10cdbb62 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:53.403569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:53.404386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.404675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.405056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.405725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:53.407098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f706e62a-773c-405c-a4f6-d03038f546d1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:53.419211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:53.420253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:53.421209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 502/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:53.428154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:53.467626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd9c3d-a7a6-47c1-8694-5922157e1be0 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.467918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd9c3d-a7a6-47c1-8694-5922157e1be0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.468402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd9c3d-a7a6-47c1-8694-5922157e1be0 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.469916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 503/1014] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:53.474794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.475084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.475561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.492071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:53.493267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:53.529491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:53.534478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce2eb7-de4a-4c91-a669-25d8ffb605db None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:53.569748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 504/1015] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:53 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:53.569993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:53.570666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.574757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.574757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.574757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:53.574757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:53.574757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:53.579526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 512/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:53.585301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:53.597721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:53.609304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:53.702919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74dc5-0ea7-4b51-bd68-68d0b68b0f9a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:53.735355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a54b5-9ce1-4917-a9c8-45bbae648f70 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:53.740601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:53.741344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.741822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.742146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.742865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:53.743675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:53.745074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:53.752303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 505/1017] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jun 1 04:08:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:53.758371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:53.776693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:53.777648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.777937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.778263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] PATH_INFO: `/v3/services/image` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.778923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:53.780421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:53.792880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:53.793933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:53.794826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 513/1018] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Jun 1 04:08:53 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:53.801644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:53.831269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:53.856116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:53.876477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e9f1-8e8c-4324-8e51-d4a67d7560d5 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:53.912841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58ec61-9d6f-4f82-8e33-4b9fbe434890 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:53.924463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:53.925134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.925417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.925695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.926435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:53.927842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:53.950687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:53.951390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:53.951680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:53.951989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:53.952697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] RBAC: Adding query filter params (name=image) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:53.953004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:53.954376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:53.959464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 514/1019] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:53.965173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:54.059611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:54.070322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392bdc-26e6-4642-b987-00d30b83eeab None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:54.079361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 506/1020] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:08:54.119502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:54.120192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.120473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.120748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.121469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:54.121786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:54.123271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:54.135215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 515/1021] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:54.136545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:54.256808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aaa83-83ad-4c5c-b21d-f10cc1805afc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:54.295450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:54.296690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.296981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.297258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.298053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] RBAC: Adding query filter params (interface=public, service_id=df9cc69b83d74074848ec662385c73db, region_id=RegionOne) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:54.298374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:54.299783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae9a55-e88b-420e-80ce-f0201efb4dca None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:54.305844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 507/1022] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:08:54 2022] GET /identity/v3/endpoints?service_id=df9cc69b83d74074848ec662385c73db&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:54.311053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:54.413526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfbac0-d5fa-4010-8e46-8a65f799e413 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.413824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfbac0-d5fa-4010-8e46-8a65f799e413 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.414016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfbac0-d5fa-4010-8e46-8a65f799e413 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.415684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 508/1023] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:54.421463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.421757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.422040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.428543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:54.431731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197ea4f-349e-490d-8cb4-c78eeff553f7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:54.444795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:54.446032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:54.470774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:54.471485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.471768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.472050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.472801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:54.474198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:54.481490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 516/1024] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:08:54 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:08:54.534839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 509/1025] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:54.558677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:54.576171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:54.680241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e618e-4172-467e-ac0a-12a964ea73bb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:54.718849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cfde2-d4cf-4379-9495-5bc1c9ab9520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:54.719674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cfde2-d4cf-4379-9495-5bc1c9ab9520 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.719959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cfde2-d4cf-4379-9495-5bc1c9ab9520 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.720233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cfde2-d4cf-4379-9495-5bc1c9ab9520 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.721419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cfde2-d4cf-4379-9495-5bc1c9ab9520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:54.722870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cfde2-d4cf-4379-9495-5bc1c9ab9520 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:54.724693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 517/1026] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:54.730210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:54.839057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fefaf-9215-40df-81b8-767c7ad6edd1 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:54.872971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be4639-c691-4d03-86ce-02c0b2161a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:54.874199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4639-c691-4d03-86ce-02c0b2161a2e None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:54.874484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4639-c691-4d03-86ce-02c0b2161a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:54.874756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be4639-c691-4d03-86ce-02c0b2161a2e None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:54.875512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4639-c691-4d03-86ce-02c0b2161a2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:54.876951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be4639-c691-4d03-86ce-02c0b2161a2e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:54.880893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:54.973705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 510/1027] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:08:54 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:56.495741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818204-831e-4f1b-915b-f15b24151f87 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:56.496266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818204-831e-4f1b-915b-f15b24151f87 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:56.496266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818204-831e-4f1b-915b-f15b24151f87 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:56.497795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 518/1028] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:56.503057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b8e2f-ed43-48e0-b036-ab12c58b4c85 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:56.503617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b8e2f-ed43-48e0-b036-ab12c58b4c85 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:56.503963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b8e2f-ed43-48e0-b036-ab12c58b4c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:56.520737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b81b8e2f-ed43-48e0-b036-ab12c58b4c85 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:56.521933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b81b8e2f-ed43-48e0-b036-ab12c58b4c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:56.591338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 511/1029] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:56.616647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818204-831e-4f1b-915b-f15b24151f87 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:56.716000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:56.743591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818204-831e-4f1b-915b-f15b24151f87 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:56.780996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:56.781823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:56.782163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:56.782446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:56.793263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:08:56.794474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5a8c6-6595-4a8f-bdbb-45fdfe9b767a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:56.794775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5a8c6-6595-4a8f-bdbb-45fdfe9b767a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:56.795060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5a8c6-6595-4a8f-bdbb-45fdfe9b767a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:56.796656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 512/1030] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:08:56.801792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:08:56.802181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:56.803096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:08:56.804612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:56.805492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:56.805492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:56.805645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 519/1031] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jun 1 04:08:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:56.812789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:56.823906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:56.825858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:56.905981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 513/1032] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:08:56.941284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:56.948723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b49c0-cc5f-4498-8ddc-3ff3a1329d9f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:56.988774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:56.989569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:56.989928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:56.990210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:56.991062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:56.991430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:56.993144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.003249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 520/1033] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:08:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:57.008517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.009522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.013890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.064377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dedd8-3bb9-466c-8aee-23e0651b1e1d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.100091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.100902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.101186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.101458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.102134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.103529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:57.115405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:57.116795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:57.117807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 514/1034] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Jun 1 04:08:56 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:57.122710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.152137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d41c5c-1d1c-4305-84b1-822cf36e6823 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.192419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.193443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.193907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.194366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.210420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.211920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:57.233787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:57.234875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:57.236227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:57.236227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:57.236227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:57.236227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:57.236452 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c8912-13d1-40dc-ac8e-ce16f1fd05d7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.236528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 521/1035] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:57.240660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.248394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.279798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.280523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.280818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.281129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.282105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:57.282308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.283799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.289415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 515/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:57.295165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.305691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.362016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9789-e32b-4dff-a4c1-eebe48f06f80 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.401569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.402286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.402576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.402855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.403620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:57.403932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.411081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed52a69-13a2-4df7-975d-0dc8bb64102e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.413674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecf5d9-3aae-4de7-9c65-15d1fd4538a2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.417561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 522/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:57.420050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe7fa-1292-4655-9717-2c9ad81b13fd None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.420860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe7fa-1292-4655-9717-2c9ad81b13fd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.421135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe7fa-1292-4655-9717-2c9ad81b13fd None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.422467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 523/1038] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:08:57.423722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffe7fa-1292-4655-9717-2c9ad81b13fd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.454580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.455257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.455580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.455862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.456584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] RBAC: Adding query filter params (type=image) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:57.456900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.458393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549acd5-cedb-415f-8ba2-fd74c2b23aff None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.464407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 516/1039] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:08:57.465724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.466019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.466301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.484515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:08:57.485797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:08:57.516892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.532444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffe7fa-1292-4655-9717-2c9ad81b13fd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.559993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.569736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 517/1040] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:08:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:08:57.571069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.586313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.587187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.587504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.587778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.595471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.596930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:08:57.612100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:57.613178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:08:57.614236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 524/1041] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:57.615389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.678676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090753b7-bcec-4014-bca7-e90a52f05ea9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.715421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.716083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.716366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.716639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.717394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.718772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.719854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a19670-10b0-4f40-9928-9a6c08d5d742 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.759554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.760351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.760639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.760920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.762098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.763611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.765267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 525/1042] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:57.766063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.818967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 518/1043] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:08:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:57.818967 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.819328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.840681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:57.879988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e52dae-6e6d-4c12-8a3d-2ec81cb247b5 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.926513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.927197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.927514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.927793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.928640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:57.928953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.930529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:57.940152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89869d-379c-48a3-9ba0-bdb3706be1bf None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:57.950393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 526/1044] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:57.956664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:57.987950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:57.989186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:57.989593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:57.989971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:57.990710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:57.992226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:58.063452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:58.067592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1d948-5683-4555-a006-e64b594c1a1e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:58.079343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 519/1045] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:08:57 2022] POST /identity/v3/users => generated 285 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:08:58.107401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:58.108681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:58.108969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:58.109269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:58.191040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:58.192478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:58.296874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:58.303344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 527/1046] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:08:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 01 04:08:58.309311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:58.397900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:58.418490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7ad0c-b48d-4a70-8fdb-6007cfc61d95 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:58.460097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:58.461239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:58.461522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:58.461802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:58.474176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:58.475663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:08:58.498086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:58.499259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:58.500538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:08:58.500538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:08:58.500538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:08:58.500538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:58.500538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 520/1047] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:08:58.506075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:58.649274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d1a06-8cd7-49fa-8f81-c6fd071fc7b8 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:58.690095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:58.690095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:58.690095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:58.690095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:58.690712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:58.691068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:58.692614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:58.711437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 528/1048] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:58.713279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:58.831282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86023027-1149-44fe-8b02-869dce8c0ea1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:58.862698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:58.868217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:58.869023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:58.869305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:58.869586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:58.876632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:58.879066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:08:58.893115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:08:58.894182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:08:58.895189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:08:58.896432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 521/1049] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:08:58.900392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:58.971842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:08:59.016746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd4bad-f881-44f4-b812-3c137b8a374e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:59.054637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:59.055335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:59.055627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:59.055908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:59.056699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:59.057013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:59.058460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c51385-3c9f-4225-a11f-9232169eb172 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:59.076239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 529/1050] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:08:59.077483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:08:59.179427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56195c-96be-4be1-968b-7eb6082e9a3e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:08:59.214719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:08:59.215411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:08:59.215703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:08:59.215985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:08:59.216825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] RBAC: Adding query filter params (role.id=8290d13fe713458f8443fa31d743e3bc, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=56036136066d412dac682273ebf65ab9) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:08:59.217142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:08:59.218718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a382d5-44b3-4ab3-b611-8635bc2607ef None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:08:59.231434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 522/1051] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:08:59 2022] GET /identity/v3/role_assignments?user.id=56036136066d412dac682273ebf65ab9&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=8290d13fe713458f8443fa31d743e3bc => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:00.169500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376fee6-6c7d-4688-8576-87a299968cc0 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.170069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376fee6-6c7d-4688-8576-87a299968cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.170069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376fee6-6c7d-4688-8576-87a299968cc0 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.171660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 530/1052] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:00.177550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9fea0-3495-4f63-9292-c0a41458e96e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.178011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9fea0-3495-4f63-9292-c0a41458e96e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.178133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9fea0-3495-4f63-9292-c0a41458e96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.195494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-74c9fea0-3495-4f63-9292-c0a41458e96e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:00.196691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-74c9fea0-3495-4f63-9292-c0a41458e96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:00.251734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:00.272997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 523/1053] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:00.295288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376fee6-6c7d-4688-8576-87a299968cc0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:00.407574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376fee6-6c7d-4688-8576-87a299968cc0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:00.426327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:00.443961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:00.444611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.444887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.445161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.486249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154624a-5155-41c4-890b-ce629754a766 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.486550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154624a-5155-41c4-890b-ce629754a766 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.486830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154624a-5155-41c4-890b-ce629754a766 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.488340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 524/1054] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:00.494060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.494355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.494645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.515710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:00.516911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:00.534411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:00.536833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:00.592506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 525/1055] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:09:00.616476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:00.651662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 531/1056] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:00.657454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:00.676009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:00.725663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45442ce-7e6a-4b68-af10-a91a262c419a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:00.744367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:00.763592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e1e72-b422-41e6-80ca-bafe9c2b9581 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:00.763901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:00.764639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.764918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.765194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.777902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:00.786370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:00.787344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:00.788376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 526/1057] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:00.793594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:00.806871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:00.806871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.806871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.806871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.818363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:00.819895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:09:00.838723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:00.839929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:00.840839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:09:00.840839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:09:00.840839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:00.840839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:00.840839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 532/1058] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:00.846818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:00.906234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96fe51-82fa-4187-858c-1456a8e2213b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:00.921006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:00.951771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:00.952454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:00.952739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:00.953014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:00.953814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:00.954116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:00.954383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95b18-0a1d-4a3c-824e-43dc1e79ad05 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:00.955540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:00.965232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 527/1059] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:00.971561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:01.004401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.004993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.005281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.005570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.006294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:01.006718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:01.008401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:01.020391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 533/1060] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:01.028848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:01.106710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e368d3-83f0-40c5-9caa-3fe3c18366f8 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:01.154258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00230ac1-20d3-4c90-abad-92d869d20057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.155845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00230ac1-20d3-4c90-abad-92d869d20057 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.156143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00230ac1-20d3-4c90-abad-92d869d20057 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.156419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00230ac1-20d3-4c90-abad-92d869d20057 None admin] PATH_INFO: `/v3/users/90015e53cb0840c3b3512e06ecd04888` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.174330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00230ac1-20d3-4c90-abad-92d869d20057 None admin] RBAC: Authorizing `identity:get_user(user_id=90015e53cb0840c3b3512e06ecd04888)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:01.175910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00230ac1-20d3-4c90-abad-92d869d20057 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:01.178005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 528/1061] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:00 2022] GET /identity/v3/users/90015e53cb0840c3b3512e06ecd04888 => generated 285 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:01.184166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:01.299592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b504-91eb-4b1b-9209-616c6c19298e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:01.339211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:01.346393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.352914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.352914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.352914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.426598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:01.428143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:01.498947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcee7b7-7ace-48c9-b457-d714b30b223c None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.499382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcee7b7-7ace-48c9-b457-d714b30b223c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.499731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcee7b7-7ace-48c9-b457-d714b30b223c None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.501237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 529/1062] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:09:01.506753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.507095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.507587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.524512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:01.525702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:01.531293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 534/1063] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:01.537414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:01.566549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:01.594202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 530/1064] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:09:01.616500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:01.639813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56095755-b8de-4e68-b962-6b8a5687900b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:01.645445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:01.674882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.675617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.675803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.676074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.677235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] RBAC: Authorizing `identity:get_project(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:01.678671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:01.680401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 535/1065] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:01 2022] GET /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:01.686804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:01.718832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfece747-d023-425a-b57e-72d9c1338a65 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:01.752673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.753499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.754030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.754303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.765121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:01.773590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:01.774695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 531/1066] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jun 1 04:09:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:01.781019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:01.783219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:01.787855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59722ec9-8a00-4f63-8534-f641992eae7d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:01.825049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.825618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.825903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.826181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.827618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29, user.id=90015e53cb0840c3b3512e06ecd04888) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:01.827937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:01.829378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:01.835067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 536/1067] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:01 2022] GET /identity/v3/role_assignments?user.id=90015e53cb0840c3b3512e06ecd04888&scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:01.870894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:01.887765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9edc9-b67b-4963-90ea-8292670eda53 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:01.927278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:01.927993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:01.928296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:01.928565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:01.929349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:01.929663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:01.931082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:01.941382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 532/1068] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:09:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:01.947617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:02.059456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bce7-0cd9-485e-8640-11ddcb54c9a7 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:02.093534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:02.094262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:02.094540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:02.094814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:02.105077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:02.106545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:09:02.125156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:02.126117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:02.127009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:09:02.127009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:09:02.127009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:02.127009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:02.127009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 537/1069] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:02.132405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:02.235690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad533d94-918c-4f1a-be95-c74a2a660d3c None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:02.269205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:02.269876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:02.270159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:02.270438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:02.271161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:02.271500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:02.272880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:02.285299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 533/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:02.291030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:02.307265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:02.393793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0a880-eff7-4853-99f8-4419330be82f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:02.432653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:02.433452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:02.433727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:02.433995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:02.441656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:02.443106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:09:02.457651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:02.458626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:02.459986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 538/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:09:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:02.464527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:02.475898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:02.581439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63439d-0d78-4dd7-b944-a74db4ac0514 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:02.620526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:02.621215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:02.621502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:02.621781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:02.625045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:02.625045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:02.625045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:02.643338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 534/1072] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:09:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:02.649083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:02.751710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:02.792746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883aa51-0d45-4814-b004-47019d4316f1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:02.846202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:02.847493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:02.847975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:02.848469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:02.863181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:02.864708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:09:02.882736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:02.883832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:02.884722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:09:02.884722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:09:02.884722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:02.884722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:02.884722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 539/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:02.890363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:02.957139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:03.000200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b4831-2fa5-461d-8eaf-ce5c2d60d71e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:03.034400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:03.035062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:03.035366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:03.035640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:03.036353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:03.036655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:03.038029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:03.049881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 535/1074] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:03.055770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:03.158268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5002d0-720b-413f-9f8a-2833639c216a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:03.185455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:03.195140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:03.195973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:03.196250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:03.196528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:03.204165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:03.205610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:09:03.219988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:03.221095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:03.222106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 540/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:03.232125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:03.328936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba90c43-900a-4bd3-b36c-e93f830f2c17 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:03.353611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:03.362426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:03.363913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:03.363913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:03.363913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:03.364417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:03.364724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:03.366136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df0b4d-84e6-4f56-9ba9-878683229174 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:03.385245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 536/1076] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:03.391923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:03.499199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111c112-6561-4ddb-bef7-16792b2b8919 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:03.532763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3e9ba-ff7b-4e86-b80c-6718a8299da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:03.533647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3e9ba-ff7b-4e86-b80c-6718a8299da8 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:03.533932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3e9ba-ff7b-4e86-b80c-6718a8299da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:03.534211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3e9ba-ff7b-4e86-b80c-6718a8299da8 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/56036136066d412dac682273ebf65ab9/roles/8290d13fe713458f8443fa31d743e3bc` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:03.536432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3e9ba-ff7b-4e86-b80c-6718a8299da8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=56036136066d412dac682273ebf65ab9, role_id=8290d13fe713458f8443fa31d743e3bc)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:03.537876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3e9ba-ff7b-4e86-b80c-6718a8299da8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:03.603473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 541/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:09:03 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/56036136066d412dac682273ebf65ab9/roles/8290d13fe713458f8443fa31d743e3bc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:09:04.093217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abe62-fc55-4c20-951e-029f74e42b68 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.093507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abe62-fc55-4c20-951e-029f74e42b68 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.093788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844abe62-fc55-4c20-951e-029f74e42b68 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.095194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 537/1078] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:04.100540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.100829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.101112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.118017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:04.119199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:04.160790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:04.185241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 542/1079] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:04.208407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abe62-fc55-4c20-951e-029f74e42b68 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:04.309300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844abe62-fc55-4c20-951e-029f74e42b68 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:04.344042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:04.344765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.345051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.345333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.355511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:04.363500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:04.364481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:04.365355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 538/1080] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:04.370652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:04.469755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9968046-0a00-47ff-a234-b64c7a8c0aa9 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:04.502449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:04.503114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.503433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.503961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.504702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:04.505008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:04.506529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:04.515009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 543/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:04.520820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:04.567070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:04.620086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54636620-f332-4c66-a4e4-41e415af5c3d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:04.654487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:04.655405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.655693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.655970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] PATH_INFO: `/v3/users/90015e53cb0840c3b3512e06ecd04888` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.657163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] RBAC: Authorizing `identity:get_user(user_id=90015e53cb0840c3b3512e06ecd04888)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:04.658604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:04.660249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 539/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:04 2022] GET /identity/v3/users/90015e53cb0840c3b3512e06ecd04888 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:04.665909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:04.708737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:04.769646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea5c9c-5f48-4300-a790-30aa71e3ff6b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:04.802646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f683c1-c80e-4ba1-a841-5138be59b4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:04.803605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f683c1-c80e-4ba1-a841-5138be59b4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.803834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f683c1-c80e-4ba1-a841-5138be59b4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.804112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f683c1-c80e-4ba1-a841-5138be59b4a7 None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.805281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f683c1-c80e-4ba1-a841-5138be59b4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:04.806714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f683c1-c80e-4ba1-a841-5138be59b4a7 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:04.808435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 544/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:04 2022] GET /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:04.814784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:04.913110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f46d5e-8c59-48ab-8f53-f5e6ada3d07b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:04.926523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:04.946232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8e6ba-6961-4226-89bf-0380feba2d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:04.947174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8e6ba-6961-4226-89bf-0380feba2d2d None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:04.947494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8e6ba-6961-4226-89bf-0380feba2d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:04.947776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8e6ba-6961-4226-89bf-0380feba2d2d None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29/users/90015e53cb0840c3b3512e06ecd04888/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:04.949723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8e6ba-6961-4226-89bf-0380feba2d2d None admin] RBAC: Authorizing `identity:create_grant(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29, user_id=90015e53cb0840c3b3512e06ecd04888, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:04.951171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8e6ba-6961-4226-89bf-0380feba2d2d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:05.019826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 540/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:09:04 2022] PUT /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29/users/90015e53cb0840c3b3512e06ecd04888/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:09:05.846831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c60dd1-0769-4095-ad59-aa86f46295df None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:05.847425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c60dd1-0769-4095-ad59-aa86f46295df None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:05.847425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c60dd1-0769-4095-ad59-aa86f46295df None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:05.848813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 545/1085] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:05.853993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:05.854400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:05.854585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:05.871514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:05.872692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:05.938880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 541/1086] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:05.962141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c60dd1-0769-4095-ad59-aa86f46295df None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:06.064103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c60dd1-0769-4095-ad59-aa86f46295df None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:06.070937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:06.098051 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:06.098912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:06.099195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:06.099507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:06.112059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:06.120886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:06.122023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:06.122874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 546/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jun 1 04:09:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:06.128970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:06.224370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f005c-b1b2-4e00-85c1-162233aaf2f2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:06.226571 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:06.258567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:06.259236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:06.259548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:06.259828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:06.260580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:06.260890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:06.262256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:06.270755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 542/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:09:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:06.276443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:06.399280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13370d6-857d-4ee6-b0ca-b952072e3511 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:06.431285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:06.432094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:06.432375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:06.432648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:06.443482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:06.444920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:09:06.462404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:06.463537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:06.464462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:09:06.464462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:09:06.464462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:06.464462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:06.464462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 547/1089] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:06.469396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:06.567037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817abdfa-51fc-4ff6-bf4a-25962d5244ae None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:06.599215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:06.599902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:06.600184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:06.600457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:06.601172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:06.601476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:06.602840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:06.615137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 543/1090] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:06.621185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:06.644128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:06.723288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f746b-66f4-4358-84ed-4a41125bb3db None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:06.757048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:06.757834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:06.758117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:06.758391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:06.765864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:06.767331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:09:06.781800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:06.782773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:06.784097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 548/1091] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jun 1 04:09:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:06.788767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:06.816043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:06.885859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9debc5e-73b4-4e29-bd18-dc63711adeb1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:06.921608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:06.922301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:06.922583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:06.922858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:06.923744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:06.924053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:06.925508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9ead6-8cd2-437d-9101-22d5538e7469 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:06.945014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 544/1092] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jun 1 04:09:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:06.950620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.030029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.050322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c814037-a3fd-4044-9804-0a3f59c29509 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.084862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.085507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.085789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.086062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.153244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.154630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.229291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.238740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe78e0-1fbe-478f-af58-d559574ee9f6 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.241867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe78e0-1fbe-478f-af58-d559574ee9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.241867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe78e0-1fbe-478f-af58-d559574ee9f6 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.241867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 545/1093] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:07.245701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.245992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.246272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.252413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 549/1094] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:07.258222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.263466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:07.264763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:07.332219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 546/1095] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:09:07.355480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.356913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2030-ff5c-4815-989b-4061b708cf0d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.370798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.392863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.393676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.393958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.394232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.404753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.406138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:09:07.424635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:07.425747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:07.426834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:09:07.426834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:09:07.426834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:09:07.426834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:07.426834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 550/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:07.431429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.458696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4866330-78b8-41e9-8c91-021aab4fe65f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.461337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.494068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.494872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.495163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.495477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.506260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:07.514022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:07.515152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 547/1097] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:07.521250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.534864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180104-e1c0-4cd4-bb49-afc3edd6d724 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.568962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.569625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.569917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.570196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.570912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:07.571221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.572645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.583860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 551/1098] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:07.589421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.595024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.619892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace5c60-ba30-4c4e-a79f-ea6a8d33a06b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.655197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.655841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.656133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.656420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.657245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:07.657717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.658848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.667525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 548/1099] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:07.673906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.682188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.692177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a8bf6-805f-49a0-8ad7-282fdacdc755 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.726455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.727246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.727564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.727843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.734975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.736454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:09:07.750476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:07.751474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:07.752571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 552/1100] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:09:07.757752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.772442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69089b46-0776-43cb-9ae6-0eb0460df4b6 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.806337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.807206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.807518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.807793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] PATH_INFO: `/v3/users/90015e53cb0840c3b3512e06ecd04888` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.808980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] RBAC: Authorizing `identity:get_user(user_id=90015e53cb0840c3b3512e06ecd04888)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.810401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.812036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 549/1101] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/users/90015e53cb0840c3b3512e06ecd04888 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:07.817647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.821156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.856160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb64ee-7012-4d4e-85fc-4fc3616c94ed None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.890956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.891648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.891931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.892211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:07.892996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:07.893309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:07.894749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:07.907632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 553/1102] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:07.913309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:07.915797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d705ce-fcdf-4cf3-8ac2-71486db2e7f2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:07.926883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:07.949974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:07.950618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:07.950899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:07.951709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:08.011735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04b2cd-ad34-4801-9a5d-b4a5fe7937eb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:08.018582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:08.020075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:08.023478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:08.045902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:08.046562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:08.046844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:08.047120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:08.048127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] RBAC: Adding query filter params (role.id=8290d13fe713458f8443fa31d743e3bc, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=56036136066d412dac682273ebf65ab9) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:08.048961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:08.050800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:08.059342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 554/1103] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/role_assignments?user.id=56036136066d412dac682273ebf65ab9&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=8290d13fe713458f8443fa31d743e3bc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:08.127516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 550/1104] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 01 04:09:08.134247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:08.214689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:08.241178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26d8cd-a03f-498a-be4b-ea9f48b4e77d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:08.277618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe48ad8-6ebf-4408-b68a-df3153480691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:08.278482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe48ad8-6ebf-4408-b68a-df3153480691 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:08.278813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe48ad8-6ebf-4408-b68a-df3153480691 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:08.279087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe48ad8-6ebf-4408-b68a-df3153480691 None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:08.280914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe48ad8-6ebf-4408-b68a-df3153480691 None admin] RBAC: Authorizing `identity:get_project(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:08.282354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe48ad8-6ebf-4408-b68a-df3153480691 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:08.284083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 555/1105] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:08 2022] GET /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:08.290303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:08.391774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576078f-6025-49a1-8c10-4685129c3f65 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:08.393734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:08.426794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:08.427502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:08.427798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:08.428485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:08.428922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29, user.id=90015e53cb0840c3b3512e06ecd04888) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:08.429240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:08.430700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7135eb-0922-43c7-a066-cb17e9eb3bf2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:08.437580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 551/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:08 2022] GET /identity/v3/role_assignments?user.id=90015e53cb0840c3b3512e06ecd04888&scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:10.702669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d11d4d-d134-4104-808e-00bd3efa5c28 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:10.703197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d11d4d-d134-4104-808e-00bd3efa5c28 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:10.703327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d11d4d-d134-4104-808e-00bd3efa5c28 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:10.704614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 556/1107] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:10.709897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:10.710313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:10.710465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:10.729000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:10.730185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:10.794017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 552/1108] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:10.821526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d11d4d-d134-4104-808e-00bd3efa5c28 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:10.921724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d11d4d-d134-4104-808e-00bd3efa5c28 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:10.954014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee0f64-5e06-4ba1-99ca-7ea432450609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:10.954790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee0f64-5e06-4ba1-99ca-7ea432450609 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:10.955072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee0f64-5e06-4ba1-99ca-7ea432450609 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:10.955373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee0f64-5e06-4ba1-99ca-7ea432450609 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:10.956522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee0f64-5e06-4ba1-99ca-7ea432450609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:10.957953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee0f64-5e06-4ba1-99ca-7ea432450609 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:10.959692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 557/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:10.965464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:11.057849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:11.061621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab185a-ccf8-4ccd-b888-256a3fc961ed None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:11.095047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841c840-58e2-4e26-a16d-8177ad826562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:11.095742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841c840-58e2-4e26-a16d-8177ad826562 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:11.096030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841c840-58e2-4e26-a16d-8177ad826562 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:11.096309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841c840-58e2-4e26-a16d-8177ad826562 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:11.097033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841c840-58e2-4e26-a16d-8177ad826562 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:11.098464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841c840-58e2-4e26-a16d-8177ad826562 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:11.185011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 553/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:09:10 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:09:13.434096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1d3b7-6b23-499e-9d4c-8c5502290af6 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:13.434648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1d3b7-6b23-499e-9d4c-8c5502290af6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:13.434648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1d3b7-6b23-499e-9d4c-8c5502290af6 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:13.436729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 558/1111] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:13.441887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:13.442299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:13.442449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:13.458858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:13.460070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:13.526058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 554/1112] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:13.550446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1d3b7-6b23-499e-9d4c-8c5502290af6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:13.563023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:13.650165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1d3b7-6b23-499e-9d4c-8c5502290af6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:13.683087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:13.683964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:13.684298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:13.684525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] PATH_INFO: `/v3/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:13.686014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] RBAC: Authorizing `identity:get_role(role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:13.687412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:13.688768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 559/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:13 2022] GET /identity/v3/roles/a1af2eba782e475681e027a5d25335c6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:13.694666 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:13.775665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:13.791396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fd8ab-8635-4d0f-8493-022622bc3820 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:13.824249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:13.825117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:13.825399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:13.825676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] PATH_INFO: `/v3/users/75fe595304c343a59d27115e3e633cef` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:13.838464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] RBAC: Authorizing `identity:get_user(user_id=75fe595304c343a59d27115e3e633cef)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:13.839997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:13.842207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 555/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:13 2022] GET /identity/v3/users/75fe595304c343a59d27115e3e633cef => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:13.847925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:13.924145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:13.944121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28011ff4-0a3d-47d4-9bf5-64c445e23401 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:13.976650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:13.977283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:13.977563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:13.977839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:14.042765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:14.044235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:14.119098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:14.140821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 560/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:14.146760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:14.242683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751f2fe-9b40-48be-a836-0c2946725b36 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:14.275345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44709-e8ce-4701-8f7c-78d18df71f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:14.276204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44709-e8ce-4701-8f7c-78d18df71f66 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:14.276487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44709-e8ce-4701-8f7c-78d18df71f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:14.276761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44709-e8ce-4701-8f7c-78d18df71f66 None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:14.277928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44709-e8ce-4701-8f7c-78d18df71f66 None admin] RBAC: Authorizing `identity:get_project(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:14.279895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44709-e8ce-4701-8f7c-78d18df71f66 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:14.281573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 556/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:14 2022] GET /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:14.287934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:14.383590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab84101-59da-4e08-b995-6678d485df46 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:14.416603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:14.417247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:14.417535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:14.417817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:14.418640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] RBAC: Adding query filter params (role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29, user.id=75fe595304c343a59d27115e3e633cef) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:14.418958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:14.420555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16f8fc-a1e6-428d-bf6e-09abcf4f1a52 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:14.425851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 561/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:14 2022] GET /identity/v3/role_assignments?user.id=75fe595304c343a59d27115e3e633cef&scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29&role.id=a1af2eba782e475681e027a5d25335c6 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:09:16.606842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1366e-0847-4656-8d87-f67ef474a414 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:16.608046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1366e-0847-4656-8d87-f67ef474a414 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:16.608046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1366e-0847-4656-8d87-f67ef474a414 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:16.609384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 557/1118] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:16.614701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:16.615063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:16.615371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:16.632300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:16.633479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:16.667665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:16.698654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 562/1119] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:16.722308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1366e-0847-4656-8d87-f67ef474a414 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:16.734975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:16.821066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1366e-0847-4656-8d87-f67ef474a414 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:16.854353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:16.855215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:16.855525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:16.855800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] PATH_INFO: `/v3/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:16.857914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] RBAC: Authorizing `identity:get_role(role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:16.859225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:16.860616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 558/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:16 2022] GET /identity/v3/roles/a1af2eba782e475681e027a5d25335c6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:16.866455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:16.961412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaeff96-cd88-4fd1-8adc-1e3d827378f2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:16.994403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:16.995262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:16.995578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:16.995854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] PATH_INFO: `/v3/users/75fe595304c343a59d27115e3e633cef` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:16.997022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] RBAC: Authorizing `identity:get_user(user_id=75fe595304c343a59d27115e3e633cef)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:16.998487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:17.000121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 563/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:16 2022] GET /identity/v3/users/75fe595304c343a59d27115e3e633cef => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:17.006195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:17.083226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:17.103929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d9b59-d00b-451d-ba82-ee5e6eff501e None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:17.137476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b318221-9e28-4fd3-8952-de7231a39645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:17.138275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b318221-9e28-4fd3-8952-de7231a39645 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:17.138592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b318221-9e28-4fd3-8952-de7231a39645 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:17.138868 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b318221-9e28-4fd3-8952-de7231a39645 None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:17.140040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b318221-9e28-4fd3-8952-de7231a39645 None admin] RBAC: Authorizing `identity:get_project(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:17.141458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b318221-9e28-4fd3-8952-de7231a39645 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:17.143103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 559/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:17 2022] GET /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:17.149003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:17.190073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:17.245291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31f4b7-f164-4b33-90ba-b584c4d56b4e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:17.278108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d241cc-02e0-41a6-bf2a-722c5c4e754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:17.279500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d241cc-02e0-41a6-bf2a-722c5c4e754b None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:17.279794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d241cc-02e0-41a6-bf2a-722c5c4e754b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:17.280071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d241cc-02e0-41a6-bf2a-722c5c4e754b None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29/users/75fe595304c343a59d27115e3e633cef/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:17.282254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d241cc-02e0-41a6-bf2a-722c5c4e754b None admin] RBAC: Authorizing `identity:create_grant(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29, user_id=75fe595304c343a59d27115e3e633cef, role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:17.283713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d241cc-02e0-41a6-bf2a-722c5c4e754b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:17.345300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 564/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:09:17 2022] PUT /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29/users/75fe595304c343a59d27115e3e633cef/roles/a1af2eba782e475681e027a5d25335c6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:09:19.546267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bae9c-c7d5-4637-930e-b8e855314c6b None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:19.546875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bae9c-c7d5-4637-930e-b8e855314c6b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:19.546875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bae9c-c7d5-4637-930e-b8e855314c6b None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:19.548208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 560/1124] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:19.553316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:19.553743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:19.553870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:19.570078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:19.571232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:19.637864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 565/1125] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:19.661410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bae9c-c7d5-4637-930e-b8e855314c6b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:19.761414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bae9c-c7d5-4637-930e-b8e855314c6b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:19.794284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:19.795756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:19.796030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:19.796352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] PATH_INFO: `/v3/roles/a1af2eba782e475681e027a5d25335c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:19.797764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] RBAC: Authorizing `identity:get_role(role_id=a1af2eba782e475681e027a5d25335c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:19.799125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:19.800507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 561/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:19 2022] GET /identity/v3/roles/a1af2eba782e475681e027a5d25335c6 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:19.806213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:19.835001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:19.901918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0156452-1128-47ea-adc7-c29976e8dacb None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:19.936989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:19.937859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:19.938146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:19.938430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] PATH_INFO: `/v3/users/75fe595304c343a59d27115e3e633cef` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:19.939639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] RBAC: Authorizing `identity:get_user(user_id=75fe595304c343a59d27115e3e633cef)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:19.941084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:19.942704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 566/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:19 2022] GET /identity/v3/users/75fe595304c343a59d27115e3e633cef => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:19.948411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:19.964251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:20.045604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade00af6-6655-433e-9d24-8b27c767ba61 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:20.078094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:20.078675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:20.078958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:20.079235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:20.144753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:20.146198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:20.159754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:20.240278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 562/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:20.246254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:20.338325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:20.342392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10429af6-3bf3-4c02-9231-b79f6d907a13 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:20.376093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161b5d-251a-4e57-b36b-ada5d01dbec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:20.376930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161b5d-251a-4e57-b36b-ada5d01dbec8 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:20.377209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161b5d-251a-4e57-b36b-ada5d01dbec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:20.377479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161b5d-251a-4e57-b36b-ada5d01dbec8 None admin] PATH_INFO: `/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:20.378878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff161b5d-251a-4e57-b36b-ada5d01dbec8 None admin] RBAC: Authorizing `identity:get_project(project_id=9e27a2ed3c0a4b7b9ee25c69c6538a29)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:20.380330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff161b5d-251a-4e57-b36b-ada5d01dbec8 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:20.381990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 567/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:20 2022] GET /identity/v3/projects/9e27a2ed3c0a4b7b9ee25c69c6538a29 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:20.388036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:20.483209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16714198-3456-4d6f-8031-2d1d0d428119 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:20.496501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:20.516248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:20.517123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:20.517673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:20.517964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:20.518815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] RBAC: Adding query filter params (role.id=a1af2eba782e475681e027a5d25335c6, scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29, user.id=75fe595304c343a59d27115e3e633cef) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:20.519138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:20.520601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63805-4dea-41f1-8fa0-75718d19f35f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:20.526397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 563/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:20 2022] GET /identity/v3/role_assignments?user.id=75fe595304c343a59d27115e3e633cef&scope.project.id=9e27a2ed3c0a4b7b9ee25c69c6538a29&role.id=a1af2eba782e475681e027a5d25335c6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:22.731276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e971c-1d0b-4d4e-a2be-59500688d86c None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:22.731801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e971c-1d0b-4d4e-a2be-59500688d86c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:22.732042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e971c-1d0b-4d4e-a2be-59500688d86c None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:22.733463 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 568/1131] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:22.738824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:22.739257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:22.739413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:22.758188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:22.759387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:22.822664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 564/1132] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:22.846485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e971c-1d0b-4d4e-a2be-59500688d86c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:22.947008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e971c-1d0b-4d4e-a2be-59500688d86c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:22.979909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846158b-ca4c-4325-b5b0-bf276616fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:22.980687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846158b-ca4c-4325-b5b0-bf276616fd00 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:22.980973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846158b-ca4c-4325-b5b0-bf276616fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:22.981252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846158b-ca4c-4325-b5b0-bf276616fd00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:22.982975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846158b-ca4c-4325-b5b0-bf276616fd00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:22.984459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846158b-ca4c-4325-b5b0-bf276616fd00 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:22.986046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 569/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:22.991773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:23.085892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c0d32-8341-49f5-8cbd-0ed3501e46e3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:23.094668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:23.118416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d1e71-ea73-4c24-a6b3-8d46a6db6d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:23.119069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d1e71-ea73-4c24-a6b3-8d46a6db6d76 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:23.119383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d1e71-ea73-4c24-a6b3-8d46a6db6d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:23.119664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d1e71-ea73-4c24-a6b3-8d46a6db6d76 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:23.120377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d1e71-ea73-4c24-a6b3-8d46a6db6d76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:23.121799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d1e71-ea73-4c24-a6b3-8d46a6db6d76 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:23.237842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 565/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:09:22 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:09:25.496564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7000-f675-4aa6-ae56-04f94d74eb7a None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:25.497095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7000-f675-4aa6-ae56-04f94d74eb7a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:25.497095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7000-f675-4aa6-ae56-04f94d74eb7a None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:25.498550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 570/1135] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:25.504198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:25.504574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:25.504761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:25.521594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:25.522776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:25.588053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 566/1136] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:25.613271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b7000-f675-4aa6-ae56-04f94d74eb7a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:25.649287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:25.715008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b7000-f675-4aa6-ae56-04f94d74eb7a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:25.746838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6863b-34c0-4b6d-b738-3f1b6979f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:25.748112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6863b-34c0-4b6d-b738-3f1b6979f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:25.748390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6863b-34c0-4b6d-b738-3f1b6979f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:25.748661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6863b-34c0-4b6d-b738-3f1b6979f5f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:25.749808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6863b-34c0-4b6d-b738-3f1b6979f5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:25.751232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6863b-34c0-4b6d-b738-3f1b6979f5f5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:25.753095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 571/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:09:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:25.759049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:25.845048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:25.854051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc9d6f-507a-4953-a2e0-e2581126219e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:25.886981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2279b7e-be1c-4d97-b79d-83d2505c0a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:25.887668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2279b7e-be1c-4d97-b79d-83d2505c0a51 None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:25.887962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2279b7e-be1c-4d97-b79d-83d2505c0a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:25.888842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2279b7e-be1c-4d97-b79d-83d2505c0a51 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:25.889548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2279b7e-be1c-4d97-b79d-83d2505c0a51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:25.890976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2279b7e-be1c-4d97-b79d-83d2505c0a51 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:25.996676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 567/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:09:25 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:09:28.267070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f648-8fae-4cd0-b3f7-59aa41726f64 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:28.267734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f648-8fae-4cd0-b3f7-59aa41726f64 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:28.267734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f648-8fae-4cd0-b3f7-59aa41726f64 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:28.268995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 572/1139] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:28.274291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:28.282226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:28.282226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:28.291877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:28.293059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:28.356963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 568/1140] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:28.380379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f648-8fae-4cd0-b3f7-59aa41726f64 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:28.477983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f648-8fae-4cd0-b3f7-59aa41726f64 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:28.479469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:28.510091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:28.510869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:28.511149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:28.511529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:28.522183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:28.530122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:28.531199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:28.532532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 573/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:28.537242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:28.610048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:28.633119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63a3ce-430b-4d69-bc91-f8837993c3e4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:28.666835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:28.667519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:28.667804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:28.668080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:28.668817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:28.669127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:28.670484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:28.678723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 569/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:28.685612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:28.781804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710d06e-7404-4329-9ecc-ecc5a248f05e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:28.815138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:28.816021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:28.816304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:28.816579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] PATH_INFO: `/v3/users/2d9548d63ec146b9bb5f2edc28813b62` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:28.829804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] RBAC: Authorizing `identity:get_user(user_id=2d9548d63ec146b9bb5f2edc28813b62)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:28.831236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:28.832878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 574/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:28 2022] GET /identity/v3/users/2d9548d63ec146b9bb5f2edc28813b62 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:28.838541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:28.938653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10aa09-af3b-4b4a-8137-0a982c8c907f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:28.970668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:28.971323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:28.971611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:28.971884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:28.980199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:29.037106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:29.038482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:29.132632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 570/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:29.138576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:29.159397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:29.234126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82a949-6e5e-4377-acbd-092aa47aa713 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:29.266797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ad58-1e76-4ac0-841a-81067fada3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:29.267660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ad58-1e76-4ac0-841a-81067fada3ca None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:29.268183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ad58-1e76-4ac0-841a-81067fada3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:29.268183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ad58-1e76-4ac0-841a-81067fada3ca None admin] PATH_INFO: `/v3/projects/10a6977c6538428689aef6db36574d51` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:29.269340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60ad58-1e76-4ac0-841a-81067fada3ca None admin] RBAC: Authorizing `identity:get_project(project_id=10a6977c6538428689aef6db36574d51)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:29.270820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60ad58-1e76-4ac0-841a-81067fada3ca None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:29.272517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 575/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:29 2022] GET /identity/v3/projects/10a6977c6538428689aef6db36574d51 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:29.278453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:29.317318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:29.375408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87b151-4d8c-47a0-b236-261a08fdf54b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:29.408529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:29.409177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:29.409462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:29.409743 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:29.410569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=10a6977c6538428689aef6db36574d51, user.id=2d9548d63ec146b9bb5f2edc28813b62) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:29.410881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:29.412494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af816a-1608-4716-aeba-bae75904c0f4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:29.417801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 571/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:29 2022] GET /identity/v3/role_assignments?user.id=2d9548d63ec146b9bb5f2edc28813b62&scope.project.id=10a6977c6538428689aef6db36574d51&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:31.632116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a88f-e123-47be-9ea1-65fd3a558764 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:31.632665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a88f-e123-47be-9ea1-65fd3a558764 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:31.632665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8a88f-e123-47be-9ea1-65fd3a558764 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:31.634028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 576/1147] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:31.639909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:31.640341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:31.640472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:31.656717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:31.657886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:31.722581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 572/1148] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 01 04:09:31.744837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a88f-e123-47be-9ea1-65fd3a558764 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:31.832486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:31.845400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8a88f-e123-47be-9ea1-65fd3a558764 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:31.879275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:31.880033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:31.880319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:31.880594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:31.890949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:31.898670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:31.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:31.901029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 577/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:31.906049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:31.936039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:32.000754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b7ef2-3f10-468e-be27-86a166446f1e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:32.032999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:32.033657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:32.033938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:32.034212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:32.034957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:32.035263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:32.036651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:32.044903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 573/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:32.050892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:32.148300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab9c6f-37af-4b58-8d16-1cebd3852846 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:32.180399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:32.181253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:32.181533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:32.181806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] PATH_INFO: `/v3/users/2d9548d63ec146b9bb5f2edc28813b62` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:32.182952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] RBAC: Authorizing `identity:get_user(user_id=2d9548d63ec146b9bb5f2edc28813b62)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:32.184409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:32.186540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 578/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:32 2022] GET /identity/v3/users/2d9548d63ec146b9bb5f2edc28813b62 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:32.192300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:32.283227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:32.289560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeb6b1-712d-47e3-b543-a74d52944a8b None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:32.322648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c4f5-0901-414f-9822-aa504c117f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:32.323603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c4f5-0901-414f-9822-aa504c117f6b None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:32.324109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c4f5-0901-414f-9822-aa504c117f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:32.324596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c4f5-0901-414f-9822-aa504c117f6b None admin] PATH_INFO: `/v3/projects/10a6977c6538428689aef6db36574d51` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:32.326671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8c4f5-0901-414f-9822-aa504c117f6b None admin] RBAC: Authorizing `identity:get_project(project_id=10a6977c6538428689aef6db36574d51)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:32.328142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8c4f5-0901-414f-9822-aa504c117f6b None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:32.329845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 574/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:32 2022] GET /identity/v3/projects/10a6977c6538428689aef6db36574d51 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:32.335708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:32.353771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:32.432801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc4a1d-3cd4-4c55-8981-95e7f77a822b None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:32.465944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f658bc7-98b2-4637-99e4-61a4ead20176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:32.466822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658bc7-98b2-4637-99e4-61a4ead20176 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:32.467104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658bc7-98b2-4637-99e4-61a4ead20176 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:32.467414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f658bc7-98b2-4637-99e4-61a4ead20176 None admin] PATH_INFO: `/v3/projects/10a6977c6538428689aef6db36574d51/users/2d9548d63ec146b9bb5f2edc28813b62/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:32.469351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658bc7-98b2-4637-99e4-61a4ead20176 None admin] RBAC: Authorizing `identity:create_grant(project_id=10a6977c6538428689aef6db36574d51, user_id=2d9548d63ec146b9bb5f2edc28813b62, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:32.470774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f658bc7-98b2-4637-99e4-61a4ead20176 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:32.529418 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 579/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:09:32 2022] PUT /identity/v3/projects/10a6977c6538428689aef6db36574d51/users/2d9548d63ec146b9bb5f2edc28813b62/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:09:34.703843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e5615-ba53-4666-85cd-4c1ea3dd8c62 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:34.704369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e5615-ba53-4666-85cd-4c1ea3dd8c62 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:34.704477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e5615-ba53-4666-85cd-4c1ea3dd8c62 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:34.705755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 575/1154] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:34.711985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:34.712413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:34.712541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:34.729227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:34.730400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:34.795212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 580/1155] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:34.817671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e5615-ba53-4666-85cd-4c1ea3dd8c62 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:34.917292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e5615-ba53-4666-85cd-4c1ea3dd8c62 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:34.950232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:34.951016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:34.951328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:34.951608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:34.961404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:34.969149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:34.970255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:34.971488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 576/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:34.976309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:34.984758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:35.073079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f3a43-24a1-4a05-8701-0949c3896639 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:35.105033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:35.105695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:35.105977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:35.106259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:35.107680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:35.107860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:35.109230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:35.117036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 581/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:35.123402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:35.147460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:35.219076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9829bf-e637-4ca3-925d-0296384c23cd None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:35.251797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:35.252909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:35.253189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:35.253463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] PATH_INFO: `/v3/users/2d9548d63ec146b9bb5f2edc28813b62` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:35.254632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] RBAC: Authorizing `identity:get_user(user_id=2d9548d63ec146b9bb5f2edc28813b62)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:35.256180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:35.257747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 577/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:35 2022] GET /identity/v3/users/2d9548d63ec146b9bb5f2edc28813b62 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:35.263437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:35.328681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:35.361754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c66f8-e7ce-4141-8cbe-a592b475ddad None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:35.395408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:35.396049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:35.396331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:35.396605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:35.460948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:35.462399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:35.524303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:35.557499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 582/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:35.563864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:35.653184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:35.663200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2cba9-7f15-452e-80b8-d8ce34b72e80 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:35.696536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bced01-7f9f-4aba-b2fe-80a339b59ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:35.697392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bced01-7f9f-4aba-b2fe-80a339b59ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:35.697670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bced01-7f9f-4aba-b2fe-80a339b59ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:35.697939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bced01-7f9f-4aba-b2fe-80a339b59ac2 None admin] PATH_INFO: `/v3/projects/10a6977c6538428689aef6db36574d51` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:35.699088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bced01-7f9f-4aba-b2fe-80a339b59ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=10a6977c6538428689aef6db36574d51)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:35.701088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bced01-7f9f-4aba-b2fe-80a339b59ac2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:35.702741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 578/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:35 2022] GET /identity/v3/projects/10a6977c6538428689aef6db36574d51 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:35.708824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:35.804238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36468586-aba0-4061-9667-e6355b615f03 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:35.836718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:35.837365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:35.837647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:35.837923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:35.838747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=10a6977c6538428689aef6db36574d51, user.id=2d9548d63ec146b9bb5f2edc28813b62) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:35.839061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:35.840534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded42f2-7cdf-416d-b1b9-751de52dc488 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:35.846799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 583/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:35 2022] GET /identity/v3/role_assignments?user.id=2d9548d63ec146b9bb5f2edc28813b62&scope.project.id=10a6977c6538428689aef6db36574d51&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:38.066512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50336-7d6d-44b1-ba77-c851f5d91a76 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:38.067050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50336-7d6d-44b1-ba77-c851f5d91a76 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:38.067050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50336-7d6d-44b1-ba77-c851f5d91a76 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:38.068462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 579/1162] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:38.073744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:38.074169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:38.074311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:38.090640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:38.091840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:38.117390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:38.157253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 584/1163] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:38.179931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50336-7d6d-44b1-ba77-c851f5d91a76 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:38.279221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50336-7d6d-44b1-ba77-c851f5d91a76 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:38.312849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:38.313491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:38.313779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:38.314055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:38.340174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:38.380611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:38.382056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:38.475575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 580/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:38.481573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:38.576271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6b24e-8604-447b-b18c-ba09fdc8929a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:38.608044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:38.608773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:38.609056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:38.609330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:38.615760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:38.617182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c1844833-0c5b-44e2-abe0-cc4cf5343961 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:38.632045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:38.633266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:38.634124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 585/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:09:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:38.639823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:38.695139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:38.735649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36252ab7-7a8d-484b-8c6a-c3121318bcad None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:38.767774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:38.768416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:38.768701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:38.768975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:38.769683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:38.769989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:38.772180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938f32-602b-4eb2-b5e7-fc0236bf2ffc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:38.780215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 581/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jun 1 04:09:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:40.976495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e5169-8ece-49e6-8dd2-32e75d4bc086 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:40.977011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e5169-8ece-49e6-8dd2-32e75d4bc086 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:40.977011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e5169-8ece-49e6-8dd2-32e75d4bc086 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:40.978337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 586/1167] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:40.983630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b22f4-bde1-4cef-b677-eace12b108c1 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:40.984066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b22f4-bde1-4cef-b677-eace12b108c1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:40.984205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b22f4-bde1-4cef-b677-eace12b108c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:41.000666 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5f7b22f4-bde1-4cef-b677-eace12b108c1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:41.001855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5f7b22f4-bde1-4cef-b677-eace12b108c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:41.066351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 582/1168] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:41.089556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e5169-8ece-49e6-8dd2-32e75d4bc086 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:41.114749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:41.190433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e5169-8ece-49e6-8dd2-32e75d4bc086 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:41.223315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d844147-334d-4735-9b43-a7c0e03733c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:41.223970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844147-334d-4735-9b43-a7c0e03733c1 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:41.224249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844147-334d-4735-9b43-a7c0e03733c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:41.224524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844147-334d-4735-9b43-a7c0e03733c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:41.225262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d844147-334d-4735-9b43-a7c0e03733c1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:41.226665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d844147-334d-4735-9b43-a7c0e03733c1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:41.296503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 587/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Jun 1 04:09:41 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:09:43.530069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4693c3-a090-49f5-b3a4-c74ebdedacd6 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:43.530595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4693c3-a090-49f5-b3a4-c74ebdedacd6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:43.530595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4693c3-a090-49f5-b3a4-c74ebdedacd6 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:43.532122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 583/1170] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:43.537426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:43.537833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:43.537984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:43.554812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:43.556039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:43.620690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 588/1171] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:43.644288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4693c3-a090-49f5-b3a4-c74ebdedacd6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:43.696602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:43.743635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4693c3-a090-49f5-b3a4-c74ebdedacd6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:43.775190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e414b84-c78c-4efc-b953-5f6eeaa8434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:43.776068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414b84-c78c-4efc-b953-5f6eeaa8434d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:43.776351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414b84-c78c-4efc-b953-5f6eeaa8434d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:43.776626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414b84-c78c-4efc-b953-5f6eeaa8434d None admin] PATH_INFO: `/v3/domains/27d6a80647b34e1c82712797350eb3db` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:43.787991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e414b84-c78c-4efc-b953-5f6eeaa8434d None admin] RBAC: Authorizing `identity:get_domain(domain_id=27d6a80647b34e1c82712797350eb3db)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:43.789402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e414b84-c78c-4efc-b953-5f6eeaa8434d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:43.791007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 584/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Jun 1 04:09:43 2022] GET /identity/v3/domains/27d6a80647b34e1c82712797350eb3db => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:43.797485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:43.811270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:43.893846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccae831-5602-46d2-8612-5ec79a156faa None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:43.927725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d9f3e-9da9-4810-ab10-21bb0bd021b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:43.928377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d9f3e-9da9-4810-ab10-21bb0bd021b5 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:43.928661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d9f3e-9da9-4810-ab10-21bb0bd021b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:43.928938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d9f3e-9da9-4810-ab10-21bb0bd021b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:43.929648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d9f3e-9da9-4810-ab10-21bb0bd021b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:43.931077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d9f3e-9da9-4810-ab10-21bb0bd021b5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:44.021917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 589/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:09:43 2022] POST /identity/v3/projects => generated 361 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:09:46.257601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dba1a5-3d3a-47cb-bd8c-716872f8de88 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:46.258161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dba1a5-3d3a-47cb-bd8c-716872f8de88 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:46.258161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dba1a5-3d3a-47cb-bd8c-716872f8de88 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:46.259543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 585/1174] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:46.264735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:46.265167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:46.265298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:46.281781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:46.282995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:46.339748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:46.347843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 590/1175] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:46.373693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dba1a5-3d3a-47cb-bd8c-716872f8de88 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:46.471094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dba1a5-3d3a-47cb-bd8c-716872f8de88 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:46.484546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:46.505322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e66ae-f753-41d9-98d4-0d6680308acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:46.506218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e66ae-f753-41d9-98d4-0d6680308acc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:46.506505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e66ae-f753-41d9-98d4-0d6680308acc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:46.506782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e66ae-f753-41d9-98d4-0d6680308acc None admin] PATH_INFO: `/v3/domains/27d6a80647b34e1c82712797350eb3db` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:46.508019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e66ae-f753-41d9-98d4-0d6680308acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=27d6a80647b34e1c82712797350eb3db)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:46.509389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e66ae-f753-41d9-98d4-0d6680308acc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:46.510976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 586/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Jun 1 04:09:46 2022] GET /identity/v3/domains/27d6a80647b34e1c82712797350eb3db => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:46.516670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:46.612986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91713b56-3330-48c8-9b92-232f14f2bfa5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:46.646599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d73b8-8a8a-47a7-a855-9fa813933212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:46.647269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d73b8-8a8a-47a7-a855-9fa813933212 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:46.647578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d73b8-8a8a-47a7-a855-9fa813933212 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:46.647855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d73b8-8a8a-47a7-a855-9fa813933212 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:46.648564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d73b8-8a8a-47a7-a855-9fa813933212 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:46.649991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d73b8-8a8a-47a7-a855-9fa813933212 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:46.758379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 591/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:09:46 2022] POST /identity/v3/users => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:09:49.003564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78b9b-a935-4246-a984-46074c050beb None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:49.004069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78b9b-a935-4246-a984-46074c050beb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:49.004069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78b9b-a935-4246-a984-46074c050beb None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:49.005419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 587/1178] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:49.010653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:49.011092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:49.011212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:49.028082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:49.029256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:49.074464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:49.094636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 592/1179] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:49.117943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e78b9b-a935-4246-a984-46074c050beb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:49.217337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e78b9b-a935-4246-a984-46074c050beb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:49.251121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:49.251939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:49.252485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:49.252763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:49.262598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:49.270306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:49.271420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:49.272638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 588/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:49.277498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:49.374729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115414a-e0ba-41b7-b324-da154f2eb68e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:49.407773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:49.408430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:49.408714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:49.408993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:49.409732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:49.410040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:49.411559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:49.419936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 593/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:49.425756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:49.467181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:49.525010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048ba38-e40b-4602-949c-b0c25f60a100 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:49.558365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:49.559224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:49.559562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:49.559840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] PATH_INFO: `/v3/users/79ffe136f65943dc8be963a8ca50e562` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:49.572808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] RBAC: Authorizing `identity:get_user(user_id=79ffe136f65943dc8be963a8ca50e562)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:49.574235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:49.575918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 589/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:49 2022] GET /identity/v3/users/79ffe136f65943dc8be963a8ca50e562 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:49.581699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:49.625763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:49.680753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d123-8a59-4e29-82b5-a8543e35032d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:49.712243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:49.712878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:49.713159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:49.713433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:49.778964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:49.780341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:49.821013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:49.876161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 594/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:49.882288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:49.976507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:49.980217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5774b4d-c404-4fea-b510-08c4e010cf7d None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:50.013651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575c6b6-4881-4849-895f-258dd716c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:50.014803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575c6b6-4881-4849-895f-258dd716c1eb None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:50.015084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575c6b6-4881-4849-895f-258dd716c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:50.015386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575c6b6-4881-4849-895f-258dd716c1eb None admin] PATH_INFO: `/v3/projects/04dee5cab6bc464f95963e8c6ad7e554` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:50.016551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575c6b6-4881-4849-895f-258dd716c1eb None admin] RBAC: Authorizing `identity:get_project(project_id=04dee5cab6bc464f95963e8c6ad7e554)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:50.017977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575c6b6-4881-4849-895f-258dd716c1eb None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:50.019669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 590/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:49 2022] GET /identity/v3/projects/04dee5cab6bc464f95963e8c6ad7e554 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:50.025773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:50.122421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f6d0e-fe9a-46e0-9a01-582545cdeca1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:50.156359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:50.157000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:50.157284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:50.157561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:50.158384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=04dee5cab6bc464f95963e8c6ad7e554, user.id=79ffe136f65943dc8be963a8ca50e562) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:50.158697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:50.160155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c5e70-90ab-481b-9667-158947f534e0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:50.165501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 595/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:50 2022] GET /identity/v3/role_assignments?user.id=79ffe136f65943dc8be963a8ca50e562&scope.project.id=04dee5cab6bc464f95963e8c6ad7e554&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:52.346453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d106502-037d-4c30-9fb7-de490786a2bd None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:52.347011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d106502-037d-4c30-9fb7-de490786a2bd None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:52.347011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d106502-037d-4c30-9fb7-de490786a2bd None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:52.348392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 591/1186] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:52.353497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:52.353897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:52.354058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:52.365437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:52.371169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:52.373116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:52.436898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 596/1187] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:52.458791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d106502-037d-4c30-9fb7-de490786a2bd None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:52.556214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d106502-037d-4c30-9fb7-de490786a2bd None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:52.587603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:52.588384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:52.588667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:52.588938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:52.599570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:52.607950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:52.609045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:09:52.610320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 592/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:52.615171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:52.711324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0678f38-4077-44c7-9f30-0262caa225a9 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:52.744344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:52.745002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:52.745287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:52.745564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:52.746298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:52.746604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:52.747999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:52.750107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:52.756462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 597/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:52.767615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:52.790028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:52.859837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de8965-5be2-4495-b4ef-b24cfccf7c41 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:52.891800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:52.892664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:52.892950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:52.893223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] PATH_INFO: `/v3/users/79ffe136f65943dc8be963a8ca50e562` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:52.894385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] RBAC: Authorizing `identity:get_user(user_id=79ffe136f65943dc8be963a8ca50e562)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:52.895873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:52.897459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 593/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:52 2022] GET /identity/v3/users/79ffe136f65943dc8be963a8ca50e562 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:52.903785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:53.002317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546ab79-578f-480f-9f86-908c61a2c889 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:53.033941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8003a-a205-450b-b897-2a55b4c92092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:53.034792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8003a-a205-450b-b897-2a55b4c92092 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:53.035074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8003a-a205-450b-b897-2a55b4c92092 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:53.035374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8003a-a205-450b-b897-2a55b4c92092 None admin] PATH_INFO: `/v3/projects/04dee5cab6bc464f95963e8c6ad7e554` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:53.036542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8003a-a205-450b-b897-2a55b4c92092 None admin] RBAC: Authorizing `identity:get_project(project_id=04dee5cab6bc464f95963e8c6ad7e554)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:53.037971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8003a-a205-450b-b897-2a55b4c92092 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:53.039670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 598/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:52 2022] GET /identity/v3/projects/04dee5cab6bc464f95963e8c6ad7e554 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:53.046030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:53.136274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:53.142646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c19f4-fcff-489e-976e-ab08ed19629a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:53.175411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc40cf8-025c-49e7-b9ea-d08fed8ac6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:53.176279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40cf8-025c-49e7-b9ea-d08fed8ac6c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:53.176558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40cf8-025c-49e7-b9ea-d08fed8ac6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:53.176832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc40cf8-025c-49e7-b9ea-d08fed8ac6c4 None admin] PATH_INFO: `/v3/projects/04dee5cab6bc464f95963e8c6ad7e554/users/79ffe136f65943dc8be963a8ca50e562/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:53.178742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc40cf8-025c-49e7-b9ea-d08fed8ac6c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=04dee5cab6bc464f95963e8c6ad7e554, user_id=79ffe136f65943dc8be963a8ca50e562, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:53.180183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc40cf8-025c-49e7-b9ea-d08fed8ac6c4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:53.246338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 594/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:09:53 2022] PUT /identity/v3/projects/04dee5cab6bc464f95963e8c6ad7e554/users/79ffe136f65943dc8be963a8ca50e562/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:09:55.460626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa7cb8-5908-424e-813a-01ed82c817a1 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:55.461181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa7cb8-5908-424e-813a-01ed82c817a1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:55.461181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa7cb8-5908-424e-813a-01ed82c817a1 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:55.462527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 599/1193] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:09:55.467791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:55.468214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:55.468350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:55.485104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:09:55.486278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:09:55.550719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 595/1194] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:09:55.573509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa7cb8-5908-424e-813a-01ed82c817a1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:55.639274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:55.674370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa7cb8-5908-424e-813a-01ed82c817a1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:55.706974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:55.707791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:55.708071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:55.708344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:55.719796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:09:55.727259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:09:55.728361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:09:55.729375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 600/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:09:55.734973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:55.816614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:55.832089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4140d-5ae8-44ad-8f9f-18a606b275f2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:55.866350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:55.867012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:55.867295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:55.867601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:55.868349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:55.868659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:55.870026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:55.878642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 596/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:55.884416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:55.981637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9cad-dc98-4860-83c4-abc527214f6d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:56.005933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:56.014625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:56.021990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:56.021990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:56.021990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] PATH_INFO: `/v3/users/79ffe136f65943dc8be963a8ca50e562` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:56.021990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] RBAC: Authorizing `identity:get_user(user_id=79ffe136f65943dc8be963a8ca50e562)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:56.021990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:56.021990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 601/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jun 1 04:09:55 2022] GET /identity/v3/users/79ffe136f65943dc8be963a8ca50e562 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:56.026056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:56.123774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5bd13-d78e-475c-85fa-eb27273633fc None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:56.155720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:56.156362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:56.156648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:56.156927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:56.171998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:56.222533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:56.224025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:56.318132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 597/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:09:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:09:56.324032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:56.420484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03d1e7-8039-4dbb-b8ea-d9cff4153198 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:56.452311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35ad47-a12a-4036-9f08-8f2426a2e308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:56.453173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35ad47-a12a-4036-9f08-8f2426a2e308 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:56.453506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35ad47-a12a-4036-9f08-8f2426a2e308 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:56.453780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35ad47-a12a-4036-9f08-8f2426a2e308 None admin] PATH_INFO: `/v3/projects/04dee5cab6bc464f95963e8c6ad7e554` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:56.454931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35ad47-a12a-4036-9f08-8f2426a2e308 None admin] RBAC: Authorizing `identity:get_project(project_id=04dee5cab6bc464f95963e8c6ad7e554)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:56.456611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35ad47-a12a-4036-9f08-8f2426a2e308 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:56.458352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 602/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jun 1 04:09:56 2022] GET /identity/v3/projects/04dee5cab6bc464f95963e8c6ad7e554 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:09:56.464363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:09:56.509430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:09:56.561170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4943f7-f54d-4821-af98-49b902a25ad3 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:09:56.593159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:09:56.593745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:09:56.594031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:09:56.594309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:09:56.595137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=04dee5cab6bc464f95963e8c6ad7e554, user.id=79ffe136f65943dc8be963a8ca50e562) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:09:56.595487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:09:56.596914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c21bc-ec3d-49ab-a443-75fdafcb14c8 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:09:56.602573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 598/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:09:56 2022] GET /identity/v3/role_assignments?user.id=79ffe136f65943dc8be963a8ca50e562&scope.project.id=04dee5cab6bc464f95963e8c6ad7e554&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:10:06.448278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=69 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:10:06.600024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=45 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:15.508188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abf7f2-3a54-4d24-9e4d-4de044aa4938 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:15.508783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abf7f2-3a54-4d24-9e4d-4de044aa4938 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:15.508783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abf7f2-3a54-4d24-9e4d-4de044aa4938 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:15.510158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 603/1201] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:15.516486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:15.516992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:15.517349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:15.537370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:15.538562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:15.610528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 599/1202] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:15.643501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abf7f2-3a54-4d24-9e4d-4de044aa4938 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:15.760156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abf7f2-3a54-4d24-9e4d-4de044aa4938 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:15.798702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:15.799522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:15.799810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:15.800093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:15.811150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:15.812682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:15.830374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:15.833251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:15.834712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:15.834712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:15.834712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:15.834712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:15.834712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 604/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:15.837884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:15.953251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf9270-8c06-4384-94b4-84ee37eb66b9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:15.983014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:15.989866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:15.990546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:15.990881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:15.991161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:15.991991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:15.992309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:15.993700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1be8-3155-4ed1-a19a-3f3182b312ed None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:16.007726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 600/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:11:16.013560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:16.070371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:16.117941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff72953-eab4-4c6d-8ddf-f3718c62d388 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:16.156404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe832691-ebf9-46f8-a51a-18b11e934cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:16.157061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe832691-ebf9-46f8-a51a-18b11e934cda None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:16.157348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe832691-ebf9-46f8-a51a-18b11e934cda None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:16.157624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe832691-ebf9-46f8-a51a-18b11e934cda None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:16.158320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe832691-ebf9-46f8-a51a-18b11e934cda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:16.159835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe832691-ebf9-46f8-a51a-18b11e934cda None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:16.259169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 605/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jun 1 04:11:16 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:11:18.589888 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb892c71-8a90-4271-aa31-815e79bf6fad None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:18.590729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb892c71-8a90-4271-aa31-815e79bf6fad None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:18.590729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb892c71-8a90-4271-aa31-815e79bf6fad None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:18.591855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 601/1206] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:18.597152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:18.600403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:18.600403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:18.616044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:18.617238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:18.690952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 606/1207] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:18.715875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb892c71-8a90-4271-aa31-815e79bf6fad None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:18.840769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb892c71-8a90-4271-aa31-815e79bf6fad None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:18.875834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:18.876911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:18.877199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:18.877479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:18.889101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:11:18.897136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:18.904210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:18.905998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 602/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:11:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:18.908438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:18.961002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:19.043657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd574-1804-454f-b322-3ca9495e5cd5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:19.084003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:19.084687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:19.084974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:19.085253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:19.086010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:19.086326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:19.087760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:19.097751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 607/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:19.103981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:19.174416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:19.231600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77050a-2c61-4ac6-998d-286f4f2f689a None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:19.269013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:19.269945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:19.270450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:19.270762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:19.285438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:19.288184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:19.317756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:19.318885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:19.320786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:19.320786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:19.320786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:19.320786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:19.320786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 603/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:19.326183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:19.441529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:19.445020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5f37-f89b-4f78-9800-d79e28bdf9ad None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:19.483765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:19.484544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:19.484826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:19.485101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:19.485812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:19.486115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:19.487503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:19.500156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 608/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:19.506304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:19.620112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec117b-53ca-4ac7-a094-18f708362cd4 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:19.622453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:19.662609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:19.663463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:19.663748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:19.664028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:19.672228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:19.673934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:11:19.827323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:19.828770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:19.829914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 604/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:19.836685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:19.941875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73c8b-824b-46bb-b5a8-003a81d5ee0e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:19.978488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:19.979173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:19.979517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:19.980318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:19.981172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:19.981475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:19.982986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:20.002842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 609/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jun 1 04:11:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:20.008640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:20.121758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ade5-2018-4ad6-93ba-009c126192eb None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:20.156755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:20.157403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:20.157688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:20.157965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:20.192991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:20.234988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:20.236472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:20.346493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 605/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:11:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:11:20.352908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:20.376920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:20.467778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e5cfd-d3e1-4c92-936f-85df4d2d63e4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:20.513315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:20.514159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:20.514435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:20.514707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:20.525266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:20.527406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:20.546424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:20.547707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:20.548758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:20.548758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:20.548758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:20.548758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:20.548758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 610/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:20.554329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:20.618481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:20.659393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276a655-3b28-4a7f-a268-c00e0b472402 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:20.695140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:20.695832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:20.696125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:20.696409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:20.697139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:20.697449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:20.698826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:20.710810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 606/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:20.717091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:20.795540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:20.828081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e08f-1313-4615-a71b-dd84009507db None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:20.864249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:20.865049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:20.867130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:20.867130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:20.872497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:20.873993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:11:20.888115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:20.889202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:20.890296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 611/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:20.895430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:21.002997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5c40f-f9aa-44ed-b072-b579463a37d2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:21.025683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:21.037388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:21.038057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:21.038345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:21.038628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:21.039461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:21.039775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:21.041261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f052e-6d2a-4eb8-80e2-c35192f34f56 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:21.056458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 607/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:21.063098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:21.169849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd467d6-e0d6-4634-9f40-28f2077a0865 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:21.199187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:21.205161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:21.210388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:21.210388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:21.210388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:21.210388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=6848ac67d15b406693e33c9dfabbbe89) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:21.210388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:21.210388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79122d9-b280-4ec5-9522-267d8bab22e9 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:21.216063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 612/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:11:21 2022] GET /identity/v3/role_assignments?user.id=6848ac67d15b406693e33c9dfabbbe89&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:23.450093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d378f91-ace6-4848-b3e3-771bb291d43a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:23.450822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d378f91-ace6-4848-b3e3-771bb291d43a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:23.450822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d378f91-ace6-4848-b3e3-771bb291d43a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:23.452058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 608/1220] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:11:23.457363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:23.457795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:23.457895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:23.475495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:23.476690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:23.543166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 613/1221] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:23.565618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d378f91-ace6-4848-b3e3-771bb291d43a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:23.673431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d378f91-ace6-4848-b3e3-771bb291d43a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:23.713822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:23.714539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:23.714822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:23.715096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:23.725174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:11:23.733022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:23.734267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:23.735397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 609/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:11:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:23.740596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:23.843890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bedeb-6c4f-4f3b-a05f-79541a5a43c2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:23.867206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:23.878926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:23.879616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:23.879898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:23.880170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:23.881018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:23.881377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:23.882754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:23.891380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 614/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:11:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:23.897361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:23.942425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:24.000245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7180c-3ee1-4498-a136-5ec0c3c18738 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:24.034267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:24.035071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:24.035381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:24.035861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:24.046800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:24.048359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:24.066510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:24.067718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:24.068752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:24.068752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:24.068752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:24.068752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:24.068752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 610/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:11:24.073687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:24.175456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35e870-fd5e-4b4f-8582-6e2e5fc18d14 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:24.210536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:24.211217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:24.211525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:24.211802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:24.212524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:24.212831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:24.214265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:24.226606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 615/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:24.232844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:24.329833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:24.336648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f200990-da8a-477f-a73b-881847e58417 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:24.371501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:24.372302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:24.372580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:24.372847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:24.380483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:24.381925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:11:24.396656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:24.397809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:24.398841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 611/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:24.404004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:24.460298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:24.526242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03be8-b59c-4792-8370-914a0578c17f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:24.562606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:24.563285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:24.563602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:24.563883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:24.564724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:24.565035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:24.566551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:24.585440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 616/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jun 1 04:11:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:24.591759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:24.697340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c67826-0e29-4858-9319-7d1848313c27 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:24.731924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:24.732720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:24.733004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:24.733282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:24.743650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:24.745150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:24.763207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:24.764404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:24.765719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:24.765719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:24.765719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:24.765719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:24.765719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 612/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:24.770299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:24.884333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038082c5-309b-4e7c-9746-43378882eb5e None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:24.887258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:24.923741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:24.924404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:24.924729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:24.925201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:24.926074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:24.926382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:24.927799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:24.939520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 617/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:24.945694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:24.955667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:25.050356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817423-1c49-4447-8eb9-c22ed119f9ef None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:25.084726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:25.085619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:25.085914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:25.086197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:25.093541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:25.094985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:11:25.109562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:25.110720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:25.111694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 613/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:25.116686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:25.225049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89124b2-25af-471b-a36e-e06133c8b753 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:25.260802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:25.261952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:25.263141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:25.263141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:25.263412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:25.263614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:25.265062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a663d-8ee3-47dc-9d4b-68e0c599ffbc None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:25.277401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 618/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:25.283683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:25.350459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:25.385372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92658c-315d-4a51-9ba3-47ebd9cd7a88 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:25.418961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703edf4-abf8-42a0-ac2f-24d01481c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:25.419895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703edf4-abf8-42a0-ac2f-24d01481c511 None admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:25.420236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703edf4-abf8-42a0-ac2f-24d01481c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:25.420524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703edf4-abf8-42a0-ac2f-24d01481c511 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/6848ac67d15b406693e33c9dfabbbe89/roles/9782e11f49ef4d56a955cb2757671190` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:25.435825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703edf4-abf8-42a0-ac2f-24d01481c511 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=6848ac67d15b406693e33c9dfabbbe89, role_id=9782e11f49ef4d56a955cb2757671190)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:25.437273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703edf4-abf8-42a0-ac2f-24d01481c511 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:25.529247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 614/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:11:25 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/6848ac67d15b406693e33c9dfabbbe89/roles/9782e11f49ef4d56a955cb2757671190 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:11:27.811740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf01c-f780-4ee4-b8d4-a147e890c4d7 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:27.813531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf01c-f780-4ee4-b8d4-a147e890c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:27.813531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf01c-f780-4ee4-b8d4-a147e890c4d7 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:27.813741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 619/1233] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:27.820131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:27.820639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:27.820957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:27.838621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:27.839885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:27.907344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 615/1234] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:27.930997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdf01c-f780-4ee4-b8d4-a147e890c4d7 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:27.958764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:28.047702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdf01c-f780-4ee4-b8d4-a147e890c4d7 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:28.083275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:28.086433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:28.087258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:28.087577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:28.099014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:11:28.107435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:28.108658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:28.109617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 620/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jun 1 04:11:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:28.114941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:28.164118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:28.218951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4a88-3c91-4e11-9ab0-70171ff1aaa2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:28.255000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:28.255704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:28.256019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:28.256301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:28.257049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] RBAC: Adding query filter params (name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:28.257359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:28.258739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:28.266781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 616/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jun 1 04:11:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:28.272644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:28.360785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:28.376800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c9899-5696-43ef-ae75-e7ab1c70084c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:28.410461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:28.411260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:28.411599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:28.411878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:28.423448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:28.424807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:28.442626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:28.443618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:28.444530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:28.444530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:28.444530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:28.444530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:28.445221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 621/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:28.452476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:28.575142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fba62-df43-4c60-94a1-74af41189caa None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:28.584381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:28.611631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:28.612338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:28.612644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:28.612912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:28.613624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:28.614073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:28.615655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:28.628515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 617/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:28.635273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:28.742285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1036-7196-4c60-99d9-b0599a8d4cb4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:28.776205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:28.777583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:28.777864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:28.778137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:28.781093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:28.786347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:28.787891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:11:28.803011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:28.804313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:28.805475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 622/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:28.810287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:28.914753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e46fd-ba7e-4dd8-87fb-b933e9743747 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:28.950749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:28.951454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:28.951761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:28.952035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:28.952867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:28.953169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:28.954595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:28.971606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 618/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jun 1 04:11:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:28.978789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:29.079895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4d685-116c-4576-8a34-f4efef4451d0 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:29.113235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:29.123815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:29.123815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:29.123815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:29.177506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:29.184633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:29.186092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:29.302398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 623/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:11:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:11:29.308988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:29.336880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:29.411833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba06fa-37fb-4e27-bc62-36f2fe849785 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:29.451206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:29.452058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:29.452346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:29.452678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:29.464085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:29.465549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:29.484638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:29.485767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:29.486800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:29.486800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:29.486800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:29.486800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:29.486800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 619/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:29.491670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:29.590119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:29.596252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef493c-7c02-4c4c-80ba-edb0572474e5 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:29.640703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:29.641441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:29.641759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:29.642000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:29.642755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:29.643063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:29.644503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:29.658293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 624/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:29.664129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:29.751336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:29.774311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25ed33-0ee3-422e-aaf2-dcb791b973fb None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:29.811348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:29.812157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:29.812442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:29.812832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:29.820369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:29.821831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:11:29.836609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:29.837806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:29.839502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 620/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:29.844696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:29.954493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18de626-0141-43f6-bdf5-1e63d7110824 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:29.990798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:29.991525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:29.991803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:29.992075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:29.992857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:29.993163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:29.994613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36332f-879b-429d-8962-7a78df3fffd3 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:30.007702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 625/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:30.014455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:30.128362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbacf7-b5bd-4e2f-9999-bffbb6c559f1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:30.170868 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:30.171578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:30.171861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:30.172141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:30.172983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] RBAC: Adding query filter params (role.id=9782e11f49ef4d56a955cb2757671190, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=6848ac67d15b406693e33c9dfabbbe89) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:30.173299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:30.174740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc659a-33a9-4250-a3e2-4b2153b080ba None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:30.182301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 621/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:11:30 2022] GET /identity/v3/role_assignments?user.id=6848ac67d15b406693e33c9dfabbbe89&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=9782e11f49ef4d56a955cb2757671190 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:32.495984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d4d47-51d7-4cca-87d7-38cab39b5aaf None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:32.496732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d4d47-51d7-4cca-87d7-38cab39b5aaf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:32.496732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d4d47-51d7-4cca-87d7-38cab39b5aaf None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:32.497978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 626/1247] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:32.503849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:32.504270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:32.504404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:32.509830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:32.523729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:32.524914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:32.590939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 622/1248] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:32.614899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d4d47-51d7-4cca-87d7-38cab39b5aaf None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:32.637515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:32.722954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d4d47-51d7-4cca-87d7-38cab39b5aaf None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:32.756110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:32.756836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:32.757118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:32.757397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:32.768033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:11:32.775776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:32.777504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:32.778438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 627/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:32.783637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:32.883633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f935914-ce05-4f4f-ab86-dae3d743743d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:32.913198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:32.918769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:32.919466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:32.919757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:32.920034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:32.920786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:32.921115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:32.922491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:32.930398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 623/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:32.936159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:33.014785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:33.037115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a473b-0977-49df-9067-e766a49079c4 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:33.069339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:33.070287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:33.070514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:33.070724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:33.082854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:33.084361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:33.101898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:33.103005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:33.104174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:33.104174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:33.104174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:33.104174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:33.104174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 628/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:33.109049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:33.210599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074a18a-a7b8-4cf1-9e4e-5989c4b256f7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:33.245295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:33.245952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:33.246238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:33.246520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:33.247241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:33.247581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:33.248992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:33.262029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 624/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:33.267850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:33.368267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793c4ad-4ce2-43ec-9251-026a9aa28249 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:33.394599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:33.402973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85955779-585c-416e-a39b-8449eb033777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:33.403808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85955779-585c-416e-a39b-8449eb033777 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:33.404096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85955779-585c-416e-a39b-8449eb033777 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:33.404426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85955779-585c-416e-a39b-8449eb033777 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:33.412077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85955779-585c-416e-a39b-8449eb033777 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:33.413528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85955779-585c-416e-a39b-8449eb033777 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-85955779-585c-416e-a39b-8449eb033777 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:11:33.427838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:33.429045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:33.430415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 629/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:33.435580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:33.491642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:33.536585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53c703-a679-493f-9610-ce54a90829c7 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:33.569705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:33.570377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:33.570664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:33.570945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:33.571826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:33.572140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:33.574136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:33.591889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 625/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jun 1 04:11:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:33.599468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85955779-585c-416e-a39b-8449eb033777 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:33.700229 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85955779-585c-416e-a39b-8449eb033777 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:33.733627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:33.734261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:33.734543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:33.734819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:33.790037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:33.802532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:33.804015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:33.901589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 630/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:11:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:11:33.907499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:34.009119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b852-918e-4b9e-be15-66f9984b7e52 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:34.023057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:34.045120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:34.045822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:34.046106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:34.046381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:34.056184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:34.058231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:34.075431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:34.076359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:34.077291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:34.077291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:34.077291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:34.077291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:34.077291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 626/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:34.082933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:34.175100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:34.187443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844d7e6-2882-4226-b279-b7af6eaa3e96 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:34.221116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:34.221787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:34.222068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:34.222350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:34.223069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:34.223405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:34.224781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:34.236538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 631/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:34.242357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:34.343014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fb198-f662-47f5-9575-dd4d6116c9b2 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:34.376806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:34.377585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:34.377866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:34.378138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:34.384576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:34.386003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:11:34.399790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:34.404969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:34.405879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 627/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:34.407680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:34.521289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0f9b1-2f37-42d8-81c0-1d07decca2b1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:34.559727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:34.560902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:34.560902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:34.561751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:34.561751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:34.562162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:34.563513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4676a3a-c0f0-4b79-91da-e88b8e4a088e None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:34.577022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 632/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:34.583180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:34.612407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:34.690653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9777d-3a81-48fc-8ffa-ab23302fec70 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:34.725250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:34.725907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:34.726192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:34.726466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:34.727326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=6848ac67d15b406693e33c9dfabbbe89) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:34.727642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:34.729061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb15a8f-d9e3-4044-b25b-9556445220be None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:34.734605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 628/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:11:34 2022] GET /identity/v3/role_assignments?user.id=6848ac67d15b406693e33c9dfabbbe89&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=02d9e6449ea143beb203824112640698 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:11:36.953367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a4779-2aba-4f0d-8875-5a67ca6c6bbf None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:36.953871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a4779-2aba-4f0d-8875-5a67ca6c6bbf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:36.953871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a4779-2aba-4f0d-8875-5a67ca6c6bbf None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:36.955246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 633/1261] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:36.960720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:36.961151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:36.961300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:36.978522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:36.979812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:37.046433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 629/1262] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:37.069511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a4779-2aba-4f0d-8875-5a67ca6c6bbf None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:37.082949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:37.174832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a4779-2aba-4f0d-8875-5a67ca6c6bbf None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:37.209208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:37.209973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:37.210256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:37.210534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:37.220917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:11:37.228567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:37.230131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:37.231251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 634/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:37.235880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:37.254815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:37.333509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe248b-84e4-4d97-9f35-edf78c2a7356 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:37.367707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:37.368373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:37.368655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:37.368928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:37.369681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:37.370489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:37.371882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:37.380450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 630/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:11:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:37.387069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:37.453793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:37.487898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c52ec-9ab1-49e7-a0da-70aedd1393eb None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:37.523168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:37.523991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:37.524276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:37.524567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:37.535220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:37.536678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:37.554420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:37.555595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:37.562500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:37.562500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:37.562500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:37.562500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:37.562500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 635/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:37.563802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:37.679794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:37.699881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0e62d-a397-4e3f-95bc-0359ccc59f15 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:37.733324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:37.734026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:37.734377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:37.734720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:37.735901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:37.735901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:37.737268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:37.750601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 631/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:37.758708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:37.878999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966fc8-2d7e-4472-8875-13d40afd59b1 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:37.902816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:37.917472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:37.918263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:37.918542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:37.918858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:37.926390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:37.927859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:11:37.943119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:37.944368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:37.945424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 636/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:37.951706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:38.053717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b693254-4373-4795-a53b-c97b17e89e79 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:38.087197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:38.087856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:38.088140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:38.088417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:38.089776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:38.090161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:38.091566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:38.100557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:38.108913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 632/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jun 1 04:11:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:38.114870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:38.218035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d830c08-e1d6-47a6-a2b2-37354f260cd6 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:38.252470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:38.253800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:38.253800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:38.253974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:38.264671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:38.266109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:38.283636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:38.284767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:38.286014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:38.286014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:38.286014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:38.286014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:38.286014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 637/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 01 04:11:38.291126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:38.391260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94410991-af8a-40f4-9403-f3c0675ff759 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:38.426115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:38.426778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:38.427118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:38.427471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:38.428189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:38.428498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:38.429862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:38.441044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 633/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:38.446906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:38.478939 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:38.547208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687cbbe-12c4-4250-8dad-7ee2e5bdbc8d None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:38.580442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:38.581232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:38.581513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:38.581788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:38.588451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:38.590445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:11:38.605620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:38.606732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:38.607990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 638/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:11:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:38.612739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:38.673559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:38.721603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12dae8-fb8e-4ebb-ae10-8b196db3c2a1 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:38.757242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:38.757904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:38.758189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:38.758466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:38.759253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:38.759594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:38.761625 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935c4e-4265-4f54-b603-15d0b13de945 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:38.773720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 634/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:11:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:38.779898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:38.873140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:38.882178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358993b-0814-4165-a457-3bb750893c05 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:38.918398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8a060-4d99-4272-9f31-c062484be889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:38.919283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8a060-4d99-4272-9f31-c062484be889 None admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:38.919600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8a060-4d99-4272-9f31-c062484be889 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:38.919879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8a060-4d99-4272-9f31-c062484be889 None admin] PATH_INFO: `/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/6848ac67d15b406693e33c9dfabbbe89/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:38.921819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8a060-4d99-4272-9f31-c062484be889 None admin] RBAC: Authorizing `identity:create_grant(project_id=c694617dec334b69a78a72cf36fe63dc, user_id=6848ac67d15b406693e33c9dfabbbe89, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:38.923322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8a060-4d99-4272-9f31-c062484be889 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:39.019808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 639/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jun 1 04:11:38 2022] PUT /identity/v3/projects/c694617dec334b69a78a72cf36fe63dc/users/6848ac67d15b406693e33c9dfabbbe89/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:11:41.305893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ddb9-87c7-49c3-a246-7205bcc1017a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:41.306468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ddb9-87c7-49c3-a246-7205bcc1017a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:41.306468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ddb9-87c7-49c3-a246-7205bcc1017a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:41.308528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 635/1274] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:41.314326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:41.314952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:41.314952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:41.331739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:41.332974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:41.400499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 640/1275] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:41.424641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ddb9-87c7-49c3-a246-7205bcc1017a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:41.527121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ddb9-87c7-49c3-a246-7205bcc1017a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:41.545095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:41.566195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:41.566955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:41.567237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:41.567551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:41.578031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 01 04:11:41.585972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:41.589662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:41.590629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 636/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jun 1 04:11:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:41.593473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:41.682246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:41.705184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a59f14-9967-4f1f-9e57-8be1803aba76 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:41.740179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:41.740850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:41.741133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:41.741406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:41.742155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:41.742457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:41.743927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:41.756173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 641/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jun 1 04:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:41.758773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:41.860822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c75851-03a4-4559-bedb-59c2392dab37 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:41.895116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:41.895957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:41.896360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:41.896641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:41.908146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:41.909612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:41.928459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:41.929618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:41.930562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:41.930562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:41.930562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:41.930562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:41.930562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 637/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:41.935535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:42.039675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555da4-0e13-4732-85d6-97ea1ce0951c None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:42.073529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:42.074216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:42.074498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:42.074773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:42.075552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:42.075866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:42.077241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:42.090691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 642/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:42.096574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:42.127562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:42.196973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8bb3-6bce-4c41-a27b-1d436e37e7fe None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:42.231769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:42.232578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:42.232863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:42.233142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:42.240229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:42.241675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 01 04:11:42.256655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:42.258286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:42.259287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 638/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:42.264447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:42.268615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:42.365447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca076d69-495d-4631-b309-2375b4c1371a None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:42.400414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:42.401043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:42.401883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:42.402166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:42.402997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:42.403331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:42.404809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:42.421880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 643/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jun 1 04:11:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:42.427712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:42.521455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:42.530992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857a3a6-4dbc-49cd-ae00-5a62e0ce7422 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:42.564991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:42.565692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:42.565976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:42.566259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:42.634838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:42.636380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:42.733329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 639/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jun 1 04:11:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 01 04:11:42.740412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:42.772036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:42.805580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:42.844096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e447c-2b25-41ae-b7c3-bbce51d1ceac None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:42.877607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:42.878396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:42.878750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:42.879041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:42.890916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:42.892473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 01 04:11:42.913553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:42.914634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:42.915594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 01 04:11:42.915594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 01 04:11:42.915594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 01 04:11:42.915594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:42.915840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 644/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:42.921569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:43.025075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c92991-db86-46e3-8150-2b662e120817 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:43.059845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:43.060567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:43.060891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:43.061175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:43.061898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:43.062208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:43.063614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:43.075265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 640/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jun 1 04:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:43.081236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:43.184397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be448e87-1fb9-4179-8e5f-8d05c83c8140 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:43.201581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:43.222346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:43.223119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:43.223428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:43.223760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:43.231025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:43.232649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 01 04:11:43.246216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:43.247282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: ERROR keystone.server.flask.application Jun 01 04:11:43.248266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 645/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jun 1 04:11:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:43.253698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:43.304170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:43.353766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8802-7807-4b61-954e-ccf403d02fbe None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:43.388338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:43.389010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:43.389299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:43.389580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:43.390366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:43.390681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:43.392133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9949c7-609c-469b-a9fb-7bd20edad85a None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:43.404575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 641/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jun 1 04:11:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:11:43.411046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:43.510639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9d1f5-ad44-459a-b493-afe3b10a226f None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:43.547834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:43.548482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:43.548769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:43.549048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:43.549878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] RBAC: Adding query filter params (role.id=02d9e6449ea143beb203824112640698, scope.project.id=c694617dec334b69a78a72cf36fe63dc, user.id=6848ac67d15b406693e33c9dfabbbe89) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:43.550191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:43.551649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bcbc6-3728-4f83-bd89-be4f6d2bf2db None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:43.557492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 646/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jun 1 04:11:43 2022] GET /identity/v3/role_assignments?user.id=6848ac67d15b406693e33c9dfabbbe89&scope.project.id=c694617dec334b69a78a72cf36fe63dc&role.id=02d9e6449ea143beb203824112640698 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:45.849383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882d76-334d-4cac-bcdd-e686c6e2adff None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:45.849931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882d76-334d-4cac-bcdd-e686c6e2adff None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:45.850039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882d76-334d-4cac-bcdd-e686c6e2adff None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:45.851357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 642/1288] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:45.857363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:45.857791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:45.857902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:45.875503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:45.876874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:45.906494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:45.944575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 647/1289] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:45.967795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882d76-334d-4cac-bcdd-e686c6e2adff None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:46.068924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882d76-334d-4cac-bcdd-e686c6e2adff None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:46.079475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:46.101614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:46.102401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:46.102684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:46.102962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:46.103650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:46.105551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:46.117323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:46.125629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:46.126780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 643/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:11:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:46.127642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:46.224861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1493466-2f5c-452d-b56e-5581ac8269b6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:46.258918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:46.259612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:46.259899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:46.260175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:46.260906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:46.261214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:46.262627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5aebd-971e-43a7-a74b-2780b0145225 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:46.267526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 648/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jun 1 04:11:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:46.273403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:46.374199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a2d1c-764d-4f35-be56-0411c06d5412 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:46.408362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:46.408962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:46.409253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:46.409536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] PATH_INFO: `/v3/services` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:46.410243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:46.410553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:46.412069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f91a7-02e6-4b91-af7f-c074108ec205 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:46.416558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 644/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jun 1 04:11:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:48.696978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdfe8c-aee3-4c37-82e8-deb70529b95d None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:48.697818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdfe8c-aee3-4c37-82e8-deb70529b95d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:48.697972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdfe8c-aee3-4c37-82e8-deb70529b95d None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:48.700192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 649/1293] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:48.706595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dedc6d-4fa3-4e18-8855-4a71ceeef5a5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:48.707138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dedc6d-4fa3-4e18-8855-4a71ceeef5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:48.707227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dedc6d-4fa3-4e18-8855-4a71ceeef5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:48.725808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-85dedc6d-4fa3-4e18-8855-4a71ceeef5a5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:48.727107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-85dedc6d-4fa3-4e18-8855-4a71ceeef5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:48.764694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:48.806720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 645/1294] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:48.831787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdfe8c-aee3-4c37-82e8-deb70529b95d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:48.865573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:48.953867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdfe8c-aee3-4c37-82e8-deb70529b95d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:49.008363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f739eee-5713-4a55-b5d3-239918c8eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:49.009475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739eee-5713-4a55-b5d3-239918c8eeb4 None admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:49.010052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739eee-5713-4a55-b5d3-239918c8eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:49.010500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f739eee-5713-4a55-b5d3-239918c8eeb4 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:49.011994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f739eee-5713-4a55-b5d3-239918c8eeb4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:49.014498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f739eee-5713-4a55-b5d3-239918c8eeb4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:49.136048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 650/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jun 1 04:11:48 2022] POST /identity/v3/services => generated 246 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:11:51.406091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5ff6-571a-408b-9a0a-7ca37936a559 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:51.406626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5ff6-571a-408b-9a0a-7ca37936a559 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:51.406761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5ff6-571a-408b-9a0a-7ca37936a559 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:51.408075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 646/1296] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:51.413667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:51.414091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:51.414229 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:51.431233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:51.434182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:51.506972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 651/1297] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:51.530062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb5ff6-571a-408b-9a0a-7ca37936a559 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:51.630350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb5ff6-571a-408b-9a0a-7ca37936a559 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:51.665186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:51.665906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:51.666187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:51.666457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:51.667386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:51.668791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:51.681162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:51.682379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:51.683801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 647/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:11:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:51.689334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:51.708235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:51.787444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff803e-9a20-44a4-90d9-5f5eb768fe46 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:51.822541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:51.823205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:51.823522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:51.823829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:51.824524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:51.824827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:51.826196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:51.831370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 652/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jun 1 04:11:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:51.837498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:51.870168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:51.938943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4db3e-8680-4aa1-b2d7-bae48fb9bf6f None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:51.972166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:51.972821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:51.973107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:51.973381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:51.974204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] RBAC: Adding query filter params (interface=public, service_id=ad1e227e48334b22a8a85680b5191556, region_id=RegionOne) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:51.974520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:51.975932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1661-be98-495d-a5b7-6b74d1f7bf2d None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:51.981628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 648/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jun 1 04:11:51 2022] GET /identity/v3/endpoints?service_id=ad1e227e48334b22a8a85680b5191556&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:11:51.987421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:52.070485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:52.087597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89d43f-80cb-4489-b2e5-166b4e432f33 None admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:52.121219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dc8cb-f69a-4d6a-9be9-3c9575b426c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:52.121879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dc8cb-f69a-4d6a-9be9-3c9575b426c4 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:52.122161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dc8cb-f69a-4d6a-9be9-3c9575b426c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:52.122436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dc8cb-f69a-4d6a-9be9-3c9575b426c4 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:52.123111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dc8cb-f69a-4d6a-9be9-3c9575b426c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:52.124509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dc8cb-f69a-4d6a-9be9-3c9575b426c4 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:52.131524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 653/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jun 1 04:11:51 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:11:54.401598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729ba9b-0083-44b9-a00f-350ebbbe01d5 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:54.402756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729ba9b-0083-44b9-a00f-350ebbbe01d5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:54.402756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729ba9b-0083-44b9-a00f-350ebbbe01d5 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:54.404058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 649/1302] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:11:54.409377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:54.409816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:54.409962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:54.426734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:11:54.427947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:11:54.497997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 654/1303] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:11:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 01 04:11:54.521487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729ba9b-0083-44b9-a00f-350ebbbe01d5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:54.633026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:54.640265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729ba9b-0083-44b9-a00f-350ebbbe01d5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:54.677416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:54.678139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:54.678424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:54.678701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:54.679401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:54.680764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 01 04:11:54.692857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 01 04:11:54.694127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.get_or_create( Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application with Lock( Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self._enter() Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application return self.creator() Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application created_value = creator( Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: ERROR keystone.server.flask.application Jun 01 04:11:54.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 650/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jun 1 04:11:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 01 04:11:54.700814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:54.780917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:11:54.805675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683fb2e-c988-4e8a-a946-67a7c1cfd02d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:54.842309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:54.843164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:54.843623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:54.844164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] PATH_INFO: `/v3/services` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:54.845049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:11:54.845361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:54.846755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f577-aa9a-4a22-b48c-2805e3b677d5 None admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:54.852277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 655/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jun 1 04:11:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:11:54.858245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:11:54.966307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d02f-9601-42c3-bfb7-518caf49c860 None admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:11:55.000803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8297905-f23a-4b6a-b7e2-c14a9f8ae8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:11:55.001726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8297905-f23a-4b6a-b7e2-c14a9f8ae8ca None admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:11:55.002012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8297905-f23a-4b6a-b7e2-c14a9f8ae8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:11:55.002287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8297905-f23a-4b6a-b7e2-c14a9f8ae8ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:11:55.003017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8297905-f23a-4b6a-b7e2-c14a9f8ae8ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:11:55.004472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8297905-f23a-4b6a-b7e2-c14a9f8ae8ca None admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:11:55.126294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 651/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jun 1 04:11:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:12:04.899709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=30 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:12:05.121495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:18:52.707666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082121-64a1-4aea-a648-7e46b8183849 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:52.708170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082121-64a1-4aea-a648-7e46b8183849 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:52.708252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082121-64a1-4aea-a648-7e46b8183849 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:52.709596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 656/1307] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:18:52.716168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318cc3c-2008-4c85-ae05-88f465e0edac None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:52.716964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318cc3c-2008-4c85-ae05-88f465e0edac None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:52.717552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318cc3c-2008-4c85-ae05-88f465e0edac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:52.813046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a318cc3c-2008-4c85-ae05-88f465e0edac None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:18:52.814252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a318cc3c-2008-4c85-ae05-88f465e0edac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:52.924375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 652/1308] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:18:52.974953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6dfd-5765-423f-8c35-7842f0017045 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:52.975269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6dfd-5765-423f-8c35-7842f0017045 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:52.975617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6dfd-5765-423f-8c35-7842f0017045 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:52.977069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 657/1309] 10.222.0.18 () {54 vars in 964 bytes} [Wed Jun 1 04:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:18:52.983487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c33c-fe9f-4e08-b6b0-c9eeb9144d5a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:52.983783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c33c-fe9f-4e08-b6b0-c9eeb9144d5a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:52.984065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c33c-fe9f-4e08-b6b0-c9eeb9144d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:53.073253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9d27c33c-fe9f-4e08-b6b0-c9eeb9144d5a None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:18:53.074453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9d27c33c-fe9f-4e08-b6b0-c9eeb9144d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:53.097011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:18:53.155246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 653/1310] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Jun 1 04:18:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:18:53.164963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6dfd-5765-423f-8c35-7842f0017045 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:18:53.167005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8ff6dfd-5765-423f-8c35-7842f0017045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:53.283432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6dfd-5765-423f-8c35-7842f0017045 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:18:53.365326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:18:53.366437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:53.366730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:53.367566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:53.369248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:53.429108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:18:53.487522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:18:53.489952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c53fa-ddd6-41d7-aa63-0e1239a8a63d service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:18:53.512766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 658/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:18:56.079762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c75b-7921-42f6-8397-cba907334b0c None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:56.080274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c75b-7921-42f6-8397-cba907334b0c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:56.080380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c75b-7921-42f6-8397-cba907334b0c None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:56.081707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 654/1312] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:18:56.087727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208115-5beb-4b03-aab4-316baa97f79b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:56.088198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208115-5beb-4b03-aab4-316baa97f79b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:56.088290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208115-5beb-4b03-aab4-316baa97f79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:56.164213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-74208115-5beb-4b03-aab4-316baa97f79b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:18:56.165476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-74208115-5beb-4b03-aab4-316baa97f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:56.268584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 659/1313] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:18:56.322433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f258d6-4a75-482f-bfba-93d4d00ac88e None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:56.322717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f258d6-4a75-482f-bfba-93d4d00ac88e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:56.322996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f258d6-4a75-482f-bfba-93d4d00ac88e None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:56.324557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 655/1314] 10.222.0.18 () {54 vars in 964 bytes} [Wed Jun 1 04:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:18:56.330586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87313c9-5d4d-4b15-92b3-8391dabd4714 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:56.330885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87313c9-5d4d-4b15-92b3-8391dabd4714 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:56.331169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87313c9-5d4d-4b15-92b3-8391dabd4714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:56.342397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:18:56.419161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e87313c9-5d4d-4b15-92b3-8391dabd4714 None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:18:56.420240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e87313c9-5d4d-4b15-92b3-8391dabd4714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:56.503186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 660/1315] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Jun 1 04:18:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:18:56.513403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f258d6-4a75-482f-bfba-93d4d00ac88e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:18:56.517814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e5f258d6-4a75-482f-bfba-93d4d00ac88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:56.644060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f258d6-4a75-482f-bfba-93d4d00ac88e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:18:56.722406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:18:56.725015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:18:56.725745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:18:56.726084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:18:56.726476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:18:56.728252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:18:56.858645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:18:56.860400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfddeb-beec-486a-9953-e38d0a1fdd63 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:18:56.885234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 656/1316] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:00.106041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80223999-09fd-4e3c-97c8-a3132166e287 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:00.106679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80223999-09fd-4e3c-97c8-a3132166e287 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:00.106679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80223999-09fd-4e3c-97c8-a3132166e287 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:00.108073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 661/1317] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:00.113882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd571ced-0aa7-493f-bd4c-afb29b65c8cb None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:00.114372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd571ced-0aa7-493f-bd4c-afb29b65c8cb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:00.114629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd571ced-0aa7-493f-bd4c-afb29b65c8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:00.184653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fd571ced-0aa7-493f-bd4c-afb29b65c8cb None None] MFA Rules not processed for user `8ab699500cea4a518ae5ff2d36b49db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:00.185856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fd571ced-0aa7-493f-bd4c-afb29b65c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:00.238983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:00.283622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 657/1318] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:00.323655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80223999-09fd-4e3c-97c8-a3132166e287 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:00.325630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-80223999-09fd-4e3c-97c8-a3132166e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:00.434703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80223999-09fd-4e3c-97c8-a3132166e287 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:00.437537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:00.516276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:00.516927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:00.517215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:00.517489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:00.519170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:00.642591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:00.644154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74f5a3-9403-466c-89bd-525d4ec7e1ec service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:00.664108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 662/1319] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:03.972611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b2e8-f764-4921-a85b-aa366088cd78 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:03.973137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b2e8-f764-4921-a85b-aa366088cd78 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:03.973137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b2e8-f764-4921-a85b-aa366088cd78 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:03.974442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 658/1320] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:03.980520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837588-f3b7-4a7c-ab59-2422194a088a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:03.980944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837588-f3b7-4a7c-ab59-2422194a088a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:03.981044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837588-f3b7-4a7c-ab59-2422194a088a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:04.020677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:04.045855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b8837588-f3b7-4a7c-ab59-2422194a088a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:04.047126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8837588-f3b7-4a7c-ab59-2422194a088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:04.148699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 663/1321] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:04.189054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b2e8-f764-4921-a85b-aa366088cd78 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:04.191159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d518b2e8-f764-4921-a85b-aa366088cd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:04.307713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b2e8-f764-4921-a85b-aa366088cd78 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:04.384200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:04.384857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:04.385150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:04.385441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:04.387099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:04.394162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:04.499495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:04.501027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02390551-dd4e-49c9-90b3-d9ac29606723 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:04.520848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 659/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:07.860643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306f35c-f0ed-4e17-ad3f-1e48fff79660 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:07.861197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306f35c-f0ed-4e17-ad3f-1e48fff79660 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:07.861197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306f35c-f0ed-4e17-ad3f-1e48fff79660 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:07.862569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 664/1323] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:07.868379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad5b9-ca1d-4e93-afb4-71dcfc96ff14 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:07.869049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad5b9-ca1d-4e93-afb4-71dcfc96ff14 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:07.869049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ad5b9-ca1d-4e93-afb4-71dcfc96ff14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:07.950282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9b5ad5b9-ca1d-4e93-afb4-71dcfc96ff14 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:07.951442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9b5ad5b9-ca1d-4e93-afb4-71dcfc96ff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:08.017103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:08.060871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 660/1324] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:08.097723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306f35c-f0ed-4e17-ad3f-1e48fff79660 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:08.099294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1306f35c-f0ed-4e17-ad3f-1e48fff79660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:08.215619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306f35c-f0ed-4e17-ad3f-1e48fff79660 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:08.229682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:08.297867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:08.298519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:08.298804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:08.299079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:08.300817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:08.580689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:08.582610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ac054-5bf9-4f1f-8c7f-7c2eee062d10 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:08.610113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 665/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:11.040942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dd7aa-9a46-48b7-a110-8ca884af7813 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:11.041522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dd7aa-9a46-48b7-a110-8ca884af7813 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:11.041522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dd7aa-9a46-48b7-a110-8ca884af7813 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:11.042917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 661/1326] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:11.048796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da22a57-454c-4f0b-a2bf-1affc7de8379 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:11.049308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da22a57-454c-4f0b-a2bf-1affc7de8379 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:11.049429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da22a57-454c-4f0b-a2bf-1affc7de8379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:11.104467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:11.127284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6da22a57-454c-4f0b-a2bf-1affc7de8379 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:11.128616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6da22a57-454c-4f0b-a2bf-1affc7de8379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:11.242897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 666/1327] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:11.280840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dd7aa-9a46-48b7-a110-8ca884af7813 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:11.282393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-059dd7aa-9a46-48b7-a110-8ca884af7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:11.403032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dd7aa-9a46-48b7-a110-8ca884af7813 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:11.483631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:11.484362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:11.484649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:11.484928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:11.486714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:11.512129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:11.629038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:11.629038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea754249-8954-4b50-9fb2-9837c97f1925 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:11.649847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 662/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:14.043907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c0964-fa8f-4138-bc39-277b5a5a62f4 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:14.044578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c0964-fa8f-4138-bc39-277b5a5a62f4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:14.044578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c0964-fa8f-4138-bc39-277b5a5a62f4 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:14.045850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 667/1329] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:14.051178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b341f3-b071-475a-bf82-7c1a5f3114e0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:14.051695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b341f3-b071-475a-bf82-7c1a5f3114e0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:14.051837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b341f3-b071-475a-bf82-7c1a5f3114e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:14.127498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-46b341f3-b071-475a-bf82-7c1a5f3114e0 None None] MFA Rules not processed for user `8ab699500cea4a518ae5ff2d36b49db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:14.128698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-46b341f3-b071-475a-bf82-7c1a5f3114e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:14.202109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:14.245611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 663/1330] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:14.283996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c0964-fa8f-4138-bc39-277b5a5a62f4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:14.285556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-675c0964-fa8f-4138-bc39-277b5a5a62f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:14.404733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c0964-fa8f-4138-bc39-277b5a5a62f4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:14.430574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:14.500753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:14.500753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:14.501110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:14.501279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:14.502948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:14.614679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:14.616152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f91708-6181-4bf0-8e42-4f2fe23c0162 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:14.637356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 668/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:17.367782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042caa5-d066-4490-8df4-ca41b8d6d2f3 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:17.368283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042caa5-d066-4490-8df4-ca41b8d6d2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:17.368283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042caa5-d066-4490-8df4-ca41b8d6d2f3 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:17.369890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 664/1332] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:17.375586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fdaa3-a44e-4aaf-8fb7-8eb3e0b8e9c9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:17.376032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fdaa3-a44e-4aaf-8fb7-8eb3e0b8e9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:17.376157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fdaa3-a44e-4aaf-8fb7-8eb3e0b8e9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:17.437857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:17.440442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5e6fdaa3-a44e-4aaf-8fb7-8eb3e0b8e9c9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:17.441642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5e6fdaa3-a44e-4aaf-8fb7-8eb3e0b8e9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:17.540714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 669/1333] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:17.579445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042caa5-d066-4490-8df4-ca41b8d6d2f3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:17.580963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6042caa5-d066-4490-8df4-ca41b8d6d2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:17.687354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042caa5-d066-4490-8df4-ca41b8d6d2f3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:17.762896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:17.763586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:17.763873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:17.764153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:17.765819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:17.796208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:17.882748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:17.884253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388370fe-0381-4583-8cee-fe7a1d1e6d17 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:17.905155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 665/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:20.189045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6a251-f44c-4b7a-b903-7589d0ad157e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:20.189594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6a251-f44c-4b7a-b903-7589d0ad157e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:20.189594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6a251-f44c-4b7a-b903-7589d0ad157e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:20.190953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 670/1335] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:19:20.196701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107438-3c0f-47ff-83f8-00da73258a0c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:20.197091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107438-3c0f-47ff-83f8-00da73258a0c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:20.197254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85107438-3c0f-47ff-83f8-00da73258a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:20.260464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-85107438-3c0f-47ff-83f8-00da73258a0c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:20.261667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-85107438-3c0f-47ff-83f8-00da73258a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:20.335186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:20.356486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 666/1336] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:20.397269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6a251-f44c-4b7a-b903-7589d0ad157e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:20.398883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-afa6a251-f44c-4b7a-b903-7589d0ad157e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:20.500697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6a251-f44c-4b7a-b903-7589d0ad157e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:20.530363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:20.576534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:20.576734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:20.576860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:20.577181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:20.579629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:20.690028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:20.691514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee4706-a4e6-4edc-8f25-c58f946412f1 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:20.709496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 671/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:23.564731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4bcc-debe-455b-a8fb-3f404bd5d8c9 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:23.565297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4bcc-debe-455b-a8fb-3f404bd5d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:23.565297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4bcc-debe-455b-a8fb-3f404bd5d8c9 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:23.566707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 667/1338] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:23.572419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289551e-f78c-453a-be9b-dd60116cddcf None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:23.572805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289551e-f78c-453a-be9b-dd60116cddcf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:23.572982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289551e-f78c-453a-be9b-dd60116cddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:23.643171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f289551e-f78c-453a-be9b-dd60116cddcf None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:23.644398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f289551e-f78c-453a-be9b-dd60116cddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:23.652643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:23.749324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 672/1339] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:23.784104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a4bcc-debe-455b-a8fb-3f404bd5d8c9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:23.786257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0e7a4bcc-debe-455b-a8fb-3f404bd5d8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:23.881597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a4bcc-debe-455b-a8fb-3f404bd5d8c9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:23.885580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b1a03-833b-4376-873e-27024118d512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:23.886234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b1a03-833b-4376-873e-27024118d512 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:23.886521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b1a03-833b-4376-873e-27024118d512 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:23.886799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b1a03-833b-4376-873e-27024118d512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:23.895768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b1a03-833b-4376-873e-27024118d512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:23.897200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b1a03-833b-4376-873e-27024118d512 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:23.910675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 668/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:28.517685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0b99c-6876-4e65-ad3a-23655290a502 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:28.518272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0b99c-6876-4e65-ad3a-23655290a502 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:28.518272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0b99c-6876-4e65-ad3a-23655290a502 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:28.519693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 673/1341] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:28.526702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc79bf-c870-4741-8386-177fa157a2d6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:28.527266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc79bf-c870-4741-8386-177fa157a2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:28.527266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc79bf-c870-4741-8386-177fa157a2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:28.557781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-55cc79bf-c870-4741-8386-177fa157a2d6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:28.559016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-55cc79bf-c870-4741-8386-177fa157a2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:28.662436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 669/1342] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:28.702127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0b99c-6876-4e65-ad3a-23655290a502 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:28.703919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0ac0b99c-6876-4e65-ad3a-23655290a502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:28.718391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0b99c-6876-4e65-ad3a-23655290a502 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:28.722311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1c830-92a8-4972-972c-24df2b2eafae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:28.723053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c830-92a8-4972-972c-24df2b2eafae service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:28.723435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c830-92a8-4972-972c-24df2b2eafae service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:28.723783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1c830-92a8-4972-972c-24df2b2eafae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:28.735445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1c830-92a8-4972-972c-24df2b2eafae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:28.736986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1c830-92a8-4972-972c-24df2b2eafae service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:28.751344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 674/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:19:32.114627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2d178-f905-442f-905d-1de407c982a6 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:32.115150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2d178-f905-442f-905d-1de407c982a6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:32.115150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2d178-f905-442f-905d-1de407c982a6 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:32.117116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 670/1344] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:32.122730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a097d6d-a7ca-48ad-84f7-20e1ab03c253 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:32.123175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a097d6d-a7ca-48ad-84f7-20e1ab03c253 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:32.123270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a097d6d-a7ca-48ad-84f7-20e1ab03c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:32.140942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0a097d6d-a7ca-48ad-84f7-20e1ab03c253 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:32.142152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0a097d6d-a7ca-48ad-84f7-20e1ab03c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:32.150075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 675/1345] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:32.188679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2d178-f905-442f-905d-1de407c982a6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:32.199248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2d178-f905-442f-905d-1de407c982a6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:32.202884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa52015-77b7-4744-af22-09c8ae18a752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:32.204032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa52015-77b7-4744-af22-09c8ae18a752 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:32.204032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa52015-77b7-4744-af22-09c8ae18a752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:32.204242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa52015-77b7-4744-af22-09c8ae18a752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:32.213506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa52015-77b7-4744-af22-09c8ae18a752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:32.214683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa52015-77b7-4744-af22-09c8ae18a752 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:32.224156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 671/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:36.700770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e67e3c-4c5e-4ed3-a0ec-c5e1ddb2b9e8 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:36.701468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e67e3c-4c5e-4ed3-a0ec-c5e1ddb2b9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:36.701824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e67e3c-4c5e-4ed3-a0ec-c5e1ddb2b9e8 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:36.703386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 676/1347] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:36.713238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467bf6-1534-4033-875b-6b3f8f5993fe None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:36.713751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467bf6-1534-4033-875b-6b3f8f5993fe None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:36.713751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467bf6-1534-4033-875b-6b3f8f5993fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:36.731652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-14467bf6-1534-4033-875b-6b3f8f5993fe None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:36.734599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-14467bf6-1534-4033-875b-6b3f8f5993fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:36.741283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 672/1348] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:36.779823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e67e3c-4c5e-4ed3-a0ec-c5e1ddb2b9e8 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:36.791386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e67e3c-4c5e-4ed3-a0ec-c5e1ddb2b9e8 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:36.795112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac8a23-8302-433f-b7f3-bdde338029cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:36.795870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac8a23-8302-433f-b7f3-bdde338029cd service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:36.796227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac8a23-8302-433f-b7f3-bdde338029cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:36.796577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac8a23-8302-433f-b7f3-bdde338029cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:36.807496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac8a23-8302-433f-b7f3-bdde338029cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:36.808956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac8a23-8302-433f-b7f3-bdde338029cd service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:36.819571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 677/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:42.872956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9d3c-4a9d-4a69-8083-a365598e9fbc None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:42.873420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9d3c-4a9d-4a69-8083-a365598e9fbc None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:42.873537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9d3c-4a9d-4a69-8083-a365598e9fbc None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:42.884121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 673/1350] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:42.886379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb2af-c80a-4ee2-820a-3edf62ee7fb2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:42.889486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb2af-c80a-4ee2-820a-3edf62ee7fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:42.889940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb2af-c80a-4ee2-820a-3edf62ee7fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:43.030152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b66eb2af-c80a-4ee2-820a-3edf62ee7fb2 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:43.031819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b66eb2af-c80a-4ee2-820a-3edf62ee7fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:43.106750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:43.150120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 678/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:19:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:46.728360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:19:47.611530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ea0c-7411-4f6e-b88e-6ed7cb9fc358 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:47.612017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ea0c-7411-4f6e-b88e-6ed7cb9fc358 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:47.612155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ea0c-7411-4f6e-b88e-6ed7cb9fc358 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:47.613637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 674/1352] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:47.618661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fdd2a-7657-479e-bec3-c26ec8af9a7a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:47.619925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fdd2a-7657-479e-bec3-c26ec8af9a7a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:47.620467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fdd2a-7657-479e-bec3-c26ec8af9a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:47.641492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-826fdd2a-7657-479e-bec3-c26ec8af9a7a None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:47.642758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-826fdd2a-7657-479e-bec3-c26ec8af9a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:47.652953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 679/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:19:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:49.176539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4ddb3-6c06-4751-bef9-ce82749e0695 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:49.177048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4ddb3-6c06-4751-bef9-ce82749e0695 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:49.177158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4ddb3-6c06-4751-bef9-ce82749e0695 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:49.178506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 675/1354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:49.183835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f785f-9359-48b7-a065-6519a2d24835 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:49.184300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f785f-9359-48b7-a065-6519a2d24835 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:49.184406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f785f-9359-48b7-a065-6519a2d24835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:49.202433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a28f785f-9359-48b7-a065-6519a2d24835 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:49.203727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a28f785f-9359-48b7-a065-6519a2d24835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:49.212993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 680/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:19:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:19:53.293264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028213-8c25-4d34-b47d-78421cabc5a8 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:53.296922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028213-8c25-4d34-b47d-78421cabc5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:53.296922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028213-8c25-4d34-b47d-78421cabc5a8 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:53.296922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 676/1356] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:53.300733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aca952-f4c1-4b05-9230-3374f6324106 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:53.301547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aca952-f4c1-4b05-9230-3374f6324106 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:53.302023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aca952-f4c1-4b05-9230-3374f6324106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:53.323845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-27aca952-f4c1-4b05-9230-3374f6324106 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:53.325157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-27aca952-f4c1-4b05-9230-3374f6324106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:53.334059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 681/1357] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:19:53.392243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839b939-b675-4875-9f57-dc5e66aa1e33 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:53.392856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839b939-b675-4875-9f57-dc5e66aa1e33 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:53.393283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839b939-b675-4875-9f57-dc5e66aa1e33 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:53.395490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 677/1358] 10.222.0.18 () {54 vars in 961 bytes} [Wed Jun 1 04:19:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:53.403488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c47ceb-cf28-46ea-8169-054efb49d5b5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:53.404798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c47ceb-cf28-46ea-8169-054efb49d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:53.405489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c47ceb-cf28-46ea-8169-054efb49d5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:53.465919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-41c47ceb-cf28-46ea-8169-054efb49d5b5 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:53.467248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-41c47ceb-cf28-46ea-8169-054efb49d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:53.558656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 682/1359] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Jun 1 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:19:53.567033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839b939-b675-4875-9f57-dc5e66aa1e33 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:53.583472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839b939-b675-4875-9f57-dc5e66aa1e33 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:53.589862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa68cb-44e9-4129-81eb-875711d4cf5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:53.590501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa68cb-44e9-4129-81eb-875711d4cf5a service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:53.590783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa68cb-44e9-4129-81eb-875711d4cf5a service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:53.591061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa68cb-44e9-4129-81eb-875711d4cf5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:53.600619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa68cb-44e9-4129-81eb-875711d4cf5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:53.602227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa68cb-44e9-4129-81eb-875711d4cf5a service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:53.611824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 678/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:56.163635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495a847-b482-4d4d-9c98-c5f9076ba7b7 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:56.164149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495a847-b482-4d4d-9c98-c5f9076ba7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:56.164233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495a847-b482-4d4d-9c98-c5f9076ba7b7 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:56.167343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 683/1361] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:56.170793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e260134-0653-46cf-ad66-fa19ec1dc41c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:56.171185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e260134-0653-46cf-ad66-fa19ec1dc41c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:56.171403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e260134-0653-46cf-ad66-fa19ec1dc41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:56.190178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-7e260134-0653-46cf-ad66-fa19ec1dc41c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:56.191408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7e260134-0653-46cf-ad66-fa19ec1dc41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:56.201137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 679/1362] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:56.246375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495a847-b482-4d4d-9c98-c5f9076ba7b7 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:56.262472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495a847-b482-4d4d-9c98-c5f9076ba7b7 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:56.268200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3fc03-bd07-4c18-912c-626270438033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:56.268989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3fc03-bd07-4c18-912c-626270438033 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:56.269341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3fc03-bd07-4c18-912c-626270438033 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:56.269714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3fc03-bd07-4c18-912c-626270438033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:56.280597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3fc03-bd07-4c18-912c-626270438033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:56.282192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3fc03-bd07-4c18-912c-626270438033 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:56.293894 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 684/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:19:59.044387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7a521-0a2a-444c-8a5e-259a63513918 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:59.044901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7a521-0a2a-444c-8a5e-259a63513918 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:59.045017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7a521-0a2a-444c-8a5e-259a63513918 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:59.046317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 680/1364] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:59.052226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200489-3c22-4f39-b94b-26f0cb9ad4f7 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:59.052680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200489-3c22-4f39-b94b-26f0cb9ad4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:59.052809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25200489-3c22-4f39-b94b-26f0cb9ad4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:59.073114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-25200489-3c22-4f39-b94b-26f0cb9ad4f7 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:59.074875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-25200489-3c22-4f39-b94b-26f0cb9ad4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:59.083154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 685/1365] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:59.137605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486777-aab8-4152-872f-05afad9e389d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:59.138014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486777-aab8-4152-872f-05afad9e389d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:59.138292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486777-aab8-4152-872f-05afad9e389d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:59.139739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 681/1366] 10.222.0.18 () {54 vars in 961 bytes} [Wed Jun 1 04:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:19:59.148969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3d839-a8d1-492d-8824-c9a745328e1f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:59.154019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3d839-a8d1-492d-8824-c9a745328e1f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:59.154019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3d839-a8d1-492d-8824-c9a745328e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:59.177897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c8c3d839-a8d1-492d-8824-c9a745328e1f None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:19:59.179085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c8c3d839-a8d1-492d-8824-c9a745328e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:19:59.191024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 686/1367] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Jun 1 04:19:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:19:59.199656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7486777-aab8-4152-872f-05afad9e389d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:19:59.218753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7486777-aab8-4152-872f-05afad9e389d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:19:59.228002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2b1f5-852d-4778-8a4f-9d82ea27c9be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:19:59.228999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2b1f5-852d-4778-8a4f-9d82ea27c9be service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:19:59.229465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2b1f5-852d-4778-8a4f-9d82ea27c9be service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:19:59.229903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2b1f5-852d-4778-8a4f-9d82ea27c9be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:19:59.241097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2b1f5-852d-4778-8a4f-9d82ea27c9be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:19:59.242730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2b1f5-852d-4778-8a4f-9d82ea27c9be service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:19:59.253313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 682/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:01.819564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbb08a-7d9f-44e5-b5b2-b303188133cc None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:01.820128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbb08a-7d9f-44e5-b5b2-b303188133cc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:01.820128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbb08a-7d9f-44e5-b5b2-b303188133cc None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:01.821504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 687/1369] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:01.827236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873899-00ac-46fb-a65f-44fa46d4c7a8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:01.827719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873899-00ac-46fb-a65f-44fa46d4c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:01.827864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873899-00ac-46fb-a65f-44fa46d4c7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:01.853803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b8873899-00ac-46fb-a65f-44fa46d4c7a8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:01.856193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b8873899-00ac-46fb-a65f-44fa46d4c7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:01.868138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 683/1370] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:01.914410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbb08a-7d9f-44e5-b5b2-b303188133cc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:01.933225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbb08a-7d9f-44e5-b5b2-b303188133cc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:01.939745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd11a1-21cf-41e8-849b-2243447bfceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:01.940414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd11a1-21cf-41e8-849b-2243447bfceb service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:01.940709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd11a1-21cf-41e8-849b-2243447bfceb service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:01.940985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd11a1-21cf-41e8-849b-2243447bfceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:01.950984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd11a1-21cf-41e8-849b-2243447bfceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:01.952600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd11a1-21cf-41e8-849b-2243447bfceb service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:01.964388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 688/1371] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:02.043710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552441b-a834-4fb2-bcf6-fae2c3a44dca None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:02.044203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552441b-a834-4fb2-bcf6-fae2c3a44dca None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:02.044616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552441b-a834-4fb2-bcf6-fae2c3a44dca None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:02.052069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 684/1372] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:02.053074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7b11-52fa-4ead-ad06-badf97b48591 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:02.053372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7b11-52fa-4ead-ad06-badf97b48591 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:02.053651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd7b11-52fa-4ead-ad06-badf97b48591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:02.073071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0cbd7b11-52fa-4ead-ad06-badf97b48591 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:02.074276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0cbd7b11-52fa-4ead-ad06-badf97b48591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:02.084930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 689/1373] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:02.368519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d864493-3ace-4a7c-af87-bf79e4ead6af None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:02.369213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d864493-3ace-4a7c-af87-bf79e4ead6af None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:02.369617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d864493-3ace-4a7c-af87-bf79e4ead6af None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:02.371117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 685/1374] 10.222.0.18 () {54 vars in 963 bytes} [Wed Jun 1 04:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:02.377768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7378-6000-4b74-bf4f-51630e4ec6a2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:02.378066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7378-6000-4b74-bf4f-51630e4ec6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:02.378352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7378-6000-4b74-bf4f-51630e4ec6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:02.437733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-da7c7378-6000-4b74-bf4f-51630e4ec6a2 None None] MFA Rules not processed for user `3f4e0e17040d41bf91cf4022dc4b4d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:02.439539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-da7c7378-6000-4b74-bf4f-51630e4ec6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:02.444887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:20:02.516712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 690/1375] 10.222.0.18 () {58 vars in 1060 bytes} [Wed Jun 1 04:20:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:02.525269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d864493-3ace-4a7c-af87-bf79e4ead6af None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:02.536418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d864493-3ace-4a7c-af87-bf79e4ead6af None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:02.541108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1101f-0131-4e03-b8c1-8fed2689dc39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:02.541759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1101f-0131-4e03-b8c1-8fed2689dc39 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:02.542040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1101f-0131-4e03-b8c1-8fed2689dc39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:02.542312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1101f-0131-4e03-b8c1-8fed2689dc39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:02.552592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1101f-0131-4e03-b8c1-8fed2689dc39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:02.554029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1101f-0131-4e03-b8c1-8fed2689dc39 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:02.565839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 686/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:04.429132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172fd7-b04e-42b7-bca4-80d86ffbe287 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:04.429672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172fd7-b04e-42b7-bca4-80d86ffbe287 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:04.429770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172fd7-b04e-42b7-bca4-80d86ffbe287 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:04.431112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 691/1377] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:04.437185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b0cf-9dc0-473e-bcef-05c14c9e3663 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:04.437621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b0cf-9dc0-473e-bcef-05c14c9e3663 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:04.437749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b0cf-9dc0-473e-bcef-05c14c9e3663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:04.456480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a6d3b0cf-9dc0-473e-bcef-05c14c9e3663 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:04.457699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a6d3b0cf-9dc0-473e-bcef-05c14c9e3663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:04.465932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 687/1378] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:04.515945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172fd7-b04e-42b7-bca4-80d86ffbe287 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:04.532049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172fd7-b04e-42b7-bca4-80d86ffbe287 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:04.539312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc1ef8-875d-4d51-9fa7-e1c7f5d45c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:04.539734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc1ef8-875d-4d51-9fa7-e1c7f5d45c2f service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:04.539944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc1ef8-875d-4d51-9fa7-e1c7f5d45c2f service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:04.540192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc1ef8-875d-4d51-9fa7-e1c7f5d45c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:04.550259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc1ef8-875d-4d51-9fa7-e1c7f5d45c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:04.551987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc1ef8-875d-4d51-9fa7-e1c7f5d45c2f service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:04.563406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 692/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:07.044302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94452437-23b1-4033-ab09-d847c3952e43 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:07.045015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94452437-23b1-4033-ab09-d847c3952e43 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:07.045015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94452437-23b1-4033-ab09-d847c3952e43 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:07.047008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 688/1380] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:07.053310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b74079-9085-4f95-b466-4d017e580ca2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:07.053776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b74079-9085-4f95-b466-4d017e580ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:07.053891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b74079-9085-4f95-b466-4d017e580ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:07.077075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f2b74079-9085-4f95-b466-4d017e580ca2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:07.078265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f2b74079-9085-4f95-b466-4d017e580ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:07.086553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 693/1381] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:07.133506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94452437-23b1-4033-ab09-d847c3952e43 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:07.148761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94452437-23b1-4033-ab09-d847c3952e43 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:07.155463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd1c86-52a2-4659-a019-c39bb6601350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:07.156118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd1c86-52a2-4659-a019-c39bb6601350 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:07.156402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd1c86-52a2-4659-a019-c39bb6601350 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:07.156677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd1c86-52a2-4659-a019-c39bb6601350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:07.167394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd1c86-52a2-4659-a019-c39bb6601350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:07.168984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd1c86-52a2-4659-a019-c39bb6601350 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:07.179932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 689/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:09.648994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9792b3-1381-4e01-9a7f-9f81261e8c65 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:09.649617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9792b3-1381-4e01-9a7f-9f81261e8c65 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:09.649617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9792b3-1381-4e01-9a7f-9f81261e8c65 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:09.650978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 694/1383] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:09.657603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377d784-ac66-454c-8c87-d8054473d320 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:09.658310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377d784-ac66-454c-8c87-d8054473d320 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:09.658310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377d784-ac66-454c-8c87-d8054473d320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:09.680869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a377d784-ac66-454c-8c87-d8054473d320 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:09.682489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a377d784-ac66-454c-8c87-d8054473d320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:09.691769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 690/1384] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:09.738303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9792b3-1381-4e01-9a7f-9f81261e8c65 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:09.757730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9792b3-1381-4e01-9a7f-9f81261e8c65 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:09.763786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01259-10e2-4300-9cae-3a2b3c0e8b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:09.764485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01259-10e2-4300-9cae-3a2b3c0e8b25 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:09.764878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01259-10e2-4300-9cae-3a2b3c0e8b25 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:09.765264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01259-10e2-4300-9cae-3a2b3c0e8b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:09.775428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01259-10e2-4300-9cae-3a2b3c0e8b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:09.777402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01259-10e2-4300-9cae-3a2b3c0e8b25 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:09.788378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 695/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:12.305200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c1dea-e333-43d1-832f-193a08905dda None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:12.306512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c1dea-e333-43d1-832f-193a08905dda None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:12.306512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c1dea-e333-43d1-832f-193a08905dda None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:12.308423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 691/1386] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:12.315379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a072416-ec82-498f-be52-6455acc8358d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:12.315844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a072416-ec82-498f-be52-6455acc8358d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:12.316184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a072416-ec82-498f-be52-6455acc8358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:12.343915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-7a072416-ec82-498f-be52-6455acc8358d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:12.345678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7a072416-ec82-498f-be52-6455acc8358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:12.357227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 696/1387] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:12.414553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c1dea-e333-43d1-832f-193a08905dda None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:12.433235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c1dea-e333-43d1-832f-193a08905dda None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:12.440119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31733a98-4019-4c22-b442-f861b116f2b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:12.440776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31733a98-4019-4c22-b442-f861b116f2b0 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:12.441061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31733a98-4019-4c22-b442-f861b116f2b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:12.441340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31733a98-4019-4c22-b442-f861b116f2b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:12.452331 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31733a98-4019-4c22-b442-f861b116f2b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:12.454842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31733a98-4019-4c22-b442-f861b116f2b0 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:12.469623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 692/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:14.963737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc94c96-24c9-4f9c-8156-22eb2ae14e8e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:14.964271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc94c96-24c9-4f9c-8156-22eb2ae14e8e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:14.964795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc94c96-24c9-4f9c-8156-22eb2ae14e8e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:14.966840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 697/1389] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:14.974668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009a4d-3874-409b-a3be-bfd62270c4f4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:14.975720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009a4d-3874-409b-a3be-bfd62270c4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:14.975720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009a4d-3874-409b-a3be-bfd62270c4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:14.995335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-39009a4d-3874-409b-a3be-bfd62270c4f4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:14.996627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-39009a4d-3874-409b-a3be-bfd62270c4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:15.005325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 693/1390] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:15.059078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc94c96-24c9-4f9c-8156-22eb2ae14e8e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:15.074179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc94c96-24c9-4f9c-8156-22eb2ae14e8e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:15.079852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5897125-34b2-4b40-8815-5acb1518e62e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:15.080532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5897125-34b2-4b40-8815-5acb1518e62e service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:15.080831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5897125-34b2-4b40-8815-5acb1518e62e service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:15.081132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5897125-34b2-4b40-8815-5acb1518e62e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:15.091018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5897125-34b2-4b40-8815-5acb1518e62e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:15.092744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5897125-34b2-4b40-8815-5acb1518e62e service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:15.104090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 698/1391] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:17.695858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab360de6-63ca-48d9-8062-b188af613797 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:17.696478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab360de6-63ca-48d9-8062-b188af613797 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:17.696478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab360de6-63ca-48d9-8062-b188af613797 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:17.697802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 694/1392] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:17.703566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39646731-deb9-47cb-a2eb-bbfd686da72a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:17.704072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39646731-deb9-47cb-a2eb-bbfd686da72a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:17.704263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39646731-deb9-47cb-a2eb-bbfd686da72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:17.722911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-39646731-deb9-47cb-a2eb-bbfd686da72a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:17.724830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-39646731-deb9-47cb-a2eb-bbfd686da72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:17.732786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 699/1393] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:17.777846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab360de6-63ca-48d9-8062-b188af613797 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:17.785926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=46 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:20:17.793200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab360de6-63ca-48d9-8062-b188af613797 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:17.798613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadc93e-d201-4600-9ea3-01524969f9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:17.799261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc93e-d201-4600-9ea3-01524969f9e1 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:17.799698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc93e-d201-4600-9ea3-01524969f9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:17.799980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc93e-d201-4600-9ea3-01524969f9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:17.811506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc93e-d201-4600-9ea3-01524969f9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:17.813437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc93e-d201-4600-9ea3-01524969f9e1 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:17.827093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 695/1394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:19.209893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3b61e-da78-4945-8ca1-d20bbdd36b7c None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:19.210459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3b61e-da78-4945-8ca1-d20bbdd36b7c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:19.210459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3b61e-da78-4945-8ca1-d20bbdd36b7c None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:19.211885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 700/1395] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:19.217476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261ff2-5e7c-461e-9fc2-52248dc37c16 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:19.217915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261ff2-5e7c-461e-9fc2-52248dc37c16 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:19.218023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69261ff2-5e7c-461e-9fc2-52248dc37c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:19.235757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-69261ff2-5e7c-461e-9fc2-52248dc37c16 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:19.237044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-69261ff2-5e7c-461e-9fc2-52248dc37c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:19.245737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 696/1396] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:19.308543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82acda-5be7-47cf-9bd5-3c310c88b85e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:19.308904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82acda-5be7-47cf-9bd5-3c310c88b85e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:19.309249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82acda-5be7-47cf-9bd5-3c310c88b85e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:19.310668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 701/1397] 10.222.0.18 () {54 vars in 962 bytes} [Wed Jun 1 04:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:19.315546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea618bb-8e18-4ff7-bac6-c7ff06ea1408 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:19.315842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea618bb-8e18-4ff7-bac6-c7ff06ea1408 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:19.316125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea618bb-8e18-4ff7-bac6-c7ff06ea1408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:19.365209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dea618bb-8e18-4ff7-bac6-c7ff06ea1408 None None] MFA Rules not processed for user `2e0959756bd54aa58afb14b9b45d29ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:19.366434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dea618bb-8e18-4ff7-bac6-c7ff06ea1408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:19.430705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 697/1398] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Jun 1 04:20:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:19.438805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82acda-5be7-47cf-9bd5-3c310c88b85e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:19.450220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82acda-5be7-47cf-9bd5-3c310c88b85e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:19.453728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effb3e2-0bd5-4889-804c-374decb32458 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:19.454364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effb3e2-0bd5-4889-804c-374decb32458 service glance] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:19.454647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effb3e2-0bd5-4889-804c-374decb32458 service glance] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:19.454920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effb3e2-0bd5-4889-804c-374decb32458 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:19.465143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effb3e2-0bd5-4889-804c-374decb32458 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:19.466174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effb3e2-0bd5-4889-804c-374decb32458 service glance] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:19.475823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 702/1399] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:20.190703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c8b7-ce3c-47b5-9729-d0ec4c1cb03e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.190997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c8b7-ce3c-47b5-9729-d0ec4c1cb03e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.191280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c8b7-ce3c-47b5-9729-d0ec4c1cb03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.239201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-35b0c8b7-ce3c-47b5-9729-d0ec4c1cb03e None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:20.240426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-35b0c8b7-ce3c-47b5-9729-d0ec4c1cb03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:20.257443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30c6bf-c5bd-4c1c-b4b8-1941a27eb4c3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.257968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30c6bf-c5bd-4c1c-b4b8-1941a27eb4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.257968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30c6bf-c5bd-4c1c-b4b8-1941a27eb4c3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.259356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 703/1400] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:20:20.264609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.264900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.265205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.283158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:20.284395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:20.292237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 704/1401] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:20:20.310767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 698/1402] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:20:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:20.337451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:20.350480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217f584-c56d-47df-ba1c-cb3b4c18d8fc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:20.355733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bbae5-f0f9-481f-be52-35a81e58e2d5 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.356028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bbae5-f0f9-481f-be52-35a81e58e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.356312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bbae5-f0f9-481f-be52-35a81e58e2d5 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.356742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:20.357391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.357673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.357779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 699/1403] 10.222.0.18 () {54 vars in 942 bytes} [Wed Jun 1 04:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:20:20.357948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.364107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf441700-c2c0-4033-a8b4-fe0a87fe2454 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.365046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf441700-c2c0-4033-a8b4-fe0a87fe2454 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.365336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf441700-c2c0-4033-a8b4-fe0a87fe2454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.368365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:20.369967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:20.379818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 705/1404] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:20.411262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:20:20.414400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-cf441700-c2c0-4033-a8b4-fe0a87fe2454 None None] MFA Rules not processed for user `400527de384349f58866dd9e39468307`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:20.415643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cf441700-c2c0-4033-a8b4-fe0a87fe2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:20.491340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 700/1405] 10.222.0.18 () {58 vars in 1039 bytes} [Wed Jun 1 04:20:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:20:20.502560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:20.514298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b7995-ca7b-4653-be7a-6776118eed4b service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:20.519327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d86c7c-f8a0-4104-9ccd-f1656c59b1c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:20.520407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d86c7c-f8a0-4104-9ccd-f1656c59b1c7 service swift] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:20.520407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d86c7c-f8a0-4104-9ccd-f1656c59b1c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:20.520617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d86c7c-f8a0-4104-9ccd-f1656c59b1c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:20.528482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d86c7c-f8a0-4104-9ccd-f1656c59b1c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:20.530050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d86c7c-f8a0-4104-9ccd-f1656c59b1c7 service swift] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:20.538371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 706/1406] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:20:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:20:23.027136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815620e7-3424-4aad-b20f-4de34c03f9eb None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:23.028150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815620e7-3424-4aad-b20f-4de34c03f9eb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:23.028442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815620e7-3424-4aad-b20f-4de34c03f9eb None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:23.029838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 701/1407] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:23.035292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdf287-1389-4196-a491-c68ae64c6683 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:23.035739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdf287-1389-4196-a491-c68ae64c6683 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:23.035882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdf287-1389-4196-a491-c68ae64c6683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:23.055181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b8fdf287-1389-4196-a491-c68ae64c6683 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:23.056402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8fdf287-1389-4196-a491-c68ae64c6683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:23.065124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 707/1408] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:23.110154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815620e7-3424-4aad-b20f-4de34c03f9eb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:23.127805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815620e7-3424-4aad-b20f-4de34c03f9eb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:23.133521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ea473-9a17-4087-9aab-e8d73306bf2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:23.134173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea473-9a17-4087-9aab-e8d73306bf2f service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:23.134459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea473-9a17-4087-9aab-e8d73306bf2f service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:23.134740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea473-9a17-4087-9aab-e8d73306bf2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:23.143601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ea473-9a17-4087-9aab-e8d73306bf2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:23.145270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ea473-9a17-4087-9aab-e8d73306bf2f service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:23.158562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 702/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:20:33.070069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=74 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:20:33.151218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=36 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:20:44.970124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b68f3-79f4-4928-be72-159d99c61a58 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:44.970688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b68f3-79f4-4928-be72-159d99c61a58 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:44.970688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b68f3-79f4-4928-be72-159d99c61a58 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:44.972075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 708/1410] 10.222.0.18 () {54 vars in 918 bytes} [Wed Jun 1 04:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:44.978088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800503-dbbc-4ea8-a7fd-48969abfdd22 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:44.978563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800503-dbbc-4ea8-a7fd-48969abfdd22 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:44.978678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41800503-dbbc-4ea8-a7fd-48969abfdd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:44.997423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-41800503-dbbc-4ea8-a7fd-48969abfdd22 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:44.998628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-41800503-dbbc-4ea8-a7fd-48969abfdd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:45.006921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 703/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jun 1 04:20:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:45.021617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702367-5249-444e-ab0a-d9f568ca0c0d None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:45.021903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702367-5249-444e-ab0a-d9f568ca0c0d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:45.022181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702367-5249-444e-ab0a-d9f568ca0c0d None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:45.023595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 709/1412] 10.222.0.18 () {54 vars in 918 bytes} [Wed Jun 1 04:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:45.028367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46c48-ee75-457d-b8d8-c3e812862831 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:45.028653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46c48-ee75-457d-b8d8-c3e812862831 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:45.028935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46c48-ee75-457d-b8d8-c3e812862831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:45.047183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9bb46c48-ee75-457d-b8d8-c3e812862831 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:45.048394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9bb46c48-ee75-457d-b8d8-c3e812862831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:45.057480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 704/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jun 1 04:20:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:47.810469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183c1b5-d0f6-4afd-bfcd-7e7d686b86a0 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:47.811045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183c1b5-d0f6-4afd-bfcd-7e7d686b86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:47.811045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183c1b5-d0f6-4afd-bfcd-7e7d686b86a0 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:47.812413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:47.818274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19febf0-9c45-4030-9ab9-50ee43585583 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:47.818667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19febf0-9c45-4030-9ab9-50ee43585583 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:47.818841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19febf0-9c45-4030-9ab9-50ee43585583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:47.837322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d19febf0-9c45-4030-9ab9-50ee43585583 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:47.842330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d19febf0-9c45-4030-9ab9-50ee43585583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:47.846809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:47.889898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183c1b5-d0f6-4afd-bfcd-7e7d686b86a0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:47.904421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183c1b5-d0f6-4afd-bfcd-7e7d686b86a0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:47.909827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b032f-17e3-4228-8e61-348f2310c800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:47.910504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b032f-17e3-4228-8e61-348f2310c800 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:47.910786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b032f-17e3-4228-8e61-348f2310c800 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:47.911056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b032f-17e3-4228-8e61-348f2310c800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:47.919259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b032f-17e3-4228-8e61-348f2310c800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:47.920867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b032f-17e3-4228-8e61-348f2310c800 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:47.930422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 711/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:48.920606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142c6e9-35e8-483e-a714-bb19b175197d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:48.921164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142c6e9-35e8-483e-a714-bb19b175197d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:48.921164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142c6e9-35e8-483e-a714-bb19b175197d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:48.922539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:48.928214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7c96-d873-4729-bcf8-34867cf90207 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:48.928663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7c96-d873-4729-bcf8-34867cf90207 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:48.928778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7c96-d873-4729-bcf8-34867cf90207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:48.947505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-93ef7c96-d873-4729-bcf8-34867cf90207 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:48.948690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-93ef7c96-d873-4729-bcf8-34867cf90207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:48.956349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:49.111959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142c6e9-35e8-483e-a714-bb19b175197d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:49.127082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142c6e9-35e8-483e-a714-bb19b175197d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:49.133195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c2b40-1380-436b-bf5c-04e5d2db1d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:49.133782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c2b40-1380-436b-bf5c-04e5d2db1d0a service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:49.134070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c2b40-1380-436b-bf5c-04e5d2db1d0a service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:49.134349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c2b40-1380-436b-bf5c-04e5d2db1d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:49.142827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c2b40-1380-436b-bf5c-04e5d2db1d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:49.146298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c2b40-1380-436b-bf5c-04e5d2db1d0a service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:49.154204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 707/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:50.089343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde29217-fce9-4c1e-933e-7e7957134047 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:50.089922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde29217-fce9-4c1e-933e-7e7957134047 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:50.090158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde29217-fce9-4c1e-933e-7e7957134047 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:50.092168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:50.097148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74d38-5a40-4532-b107-44600f2c86ea None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:50.098185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74d38-5a40-4532-b107-44600f2c86ea None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:50.098291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74d38-5a40-4532-b107-44600f2c86ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:50.116977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-26a74d38-5a40-4532-b107-44600f2c86ea None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:50.118456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-26a74d38-5a40-4532-b107-44600f2c86ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:50.126325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:50.277620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde29217-fce9-4c1e-933e-7e7957134047 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:50.293025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde29217-fce9-4c1e-933e-7e7957134047 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:50.298312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201ed17-01e7-418d-92b8-4fbf5518c2d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:50.298944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201ed17-01e7-418d-92b8-4fbf5518c2d2 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:50.299226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201ed17-01e7-418d-92b8-4fbf5518c2d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:50.299532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201ed17-01e7-418d-92b8-4fbf5518c2d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:50.307740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201ed17-01e7-418d-92b8-4fbf5518c2d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:50.309533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201ed17-01e7-418d-92b8-4fbf5518c2d2 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:50.319107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 714/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:20:51.227064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8f44-c04a-4aea-b411-e51d54091f55 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:51.227641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8f44-c04a-4aea-b411-e51d54091f55 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:51.227641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8f44-c04a-4aea-b411-e51d54091f55 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:51.228941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:51.233905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b3ed1-961b-4e00-83ec-b4d98a2e8b84 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:51.234347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b3ed1-961b-4e00-83ec-b4d98a2e8b84 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:51.234456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b3ed1-961b-4e00-83ec-b4d98a2e8b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:51.253194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0b8b3ed1-961b-4e00-83ec-b4d98a2e8b84 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:51.255030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0b8b3ed1-961b-4e00-83ec-b4d98a2e8b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:51.262706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:51.414576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8f44-c04a-4aea-b411-e51d54091f55 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:51.430636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8f44-c04a-4aea-b411-e51d54091f55 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:51.436071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbaf91-dfd1-4c7a-8512-d7c4b5eb9f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:51.436833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbaf91-dfd1-4c7a-8512-d7c4b5eb9f8f service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:51.437000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbaf91-dfd1-4c7a-8512-d7c4b5eb9f8f service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:51.437280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbaf91-dfd1-4c7a-8512-d7c4b5eb9f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:51.446118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbaf91-dfd1-4c7a-8512-d7c4b5eb9f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:51.447747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbaf91-dfd1-4c7a-8512-d7c4b5eb9f8f service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:51.457293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 710/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:52.377924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba295-5974-4ec7-8599-470587fc0cc0 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:52.378464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba295-5974-4ec7-8599-470587fc0cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:52.378731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ba295-5974-4ec7-8599-470587fc0cc0 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:52.381361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:52.386409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acf771-92f6-41d7-9e04-1f9bd698a3ff None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:52.386821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acf771-92f6-41d7-9e04-1f9bd698a3ff None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:52.386981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acf771-92f6-41d7-9e04-1f9bd698a3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:52.406211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-02acf771-92f6-41d7-9e04-1f9bd698a3ff None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:52.407457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-02acf771-92f6-41d7-9e04-1f9bd698a3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:52.415328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:52.563694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba295-5974-4ec7-8599-470587fc0cc0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:52.578696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ba295-5974-4ec7-8599-470587fc0cc0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:52.584202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556b1d5-0ad0-4b4c-bc6a-553d5717a565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:52.584830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556b1d5-0ad0-4b4c-bc6a-553d5717a565 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:52.585118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556b1d5-0ad0-4b4c-bc6a-553d5717a565 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:52.585396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556b1d5-0ad0-4b4c-bc6a-553d5717a565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:52.593513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556b1d5-0ad0-4b4c-bc6a-553d5717a565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:52.595109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556b1d5-0ad0-4b4c-bc6a-553d5717a565 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:52.604612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 717/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:53.637668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f94bc-ed20-42eb-aba6-4c7bb4520694 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:53.638228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f94bc-ed20-42eb-aba6-4c7bb4520694 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:53.638228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f94bc-ed20-42eb-aba6-4c7bb4520694 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:53.639604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:53.644613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415f596-a0e4-49eb-93ca-1d1908eb653a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:53.645035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415f596-a0e4-49eb-93ca-1d1908eb653a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:53.645178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415f596-a0e4-49eb-93ca-1d1908eb653a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:53.663415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d415f596-a0e4-49eb-93ca-1d1908eb653a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:53.664613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d415f596-a0e4-49eb-93ca-1d1908eb653a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:53.672477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:53.825551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f94bc-ed20-42eb-aba6-4c7bb4520694 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:53.840550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f94bc-ed20-42eb-aba6-4c7bb4520694 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:53.845808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484773a1-8769-436e-9fd3-6498cb8defce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:53.846466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484773a1-8769-436e-9fd3-6498cb8defce service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:53.846750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484773a1-8769-436e-9fd3-6498cb8defce service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:53.847026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484773a1-8769-436e-9fd3-6498cb8defce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:53.855916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484773a1-8769-436e-9fd3-6498cb8defce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:53.857491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484773a1-8769-436e-9fd3-6498cb8defce service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:53.867674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 713/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:54.798433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74297-78fc-4c3b-b9b0-05d98c46a71b None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:54.799013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74297-78fc-4c3b-b9b0-05d98c46a71b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:54.799013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74297-78fc-4c3b-b9b0-05d98c46a71b None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:54.800537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:54.806034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8420a80-5717-470e-af78-e1299a0a04a6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:54.806500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8420a80-5717-470e-af78-e1299a0a04a6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:54.806612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8420a80-5717-470e-af78-e1299a0a04a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:54.826173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f8420a80-5717-470e-af78-e1299a0a04a6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:54.827405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f8420a80-5717-470e-af78-e1299a0a04a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:54.835341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:54.983808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74297-78fc-4c3b-b9b0-05d98c46a71b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:54.999044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74297-78fc-4c3b-b9b0-05d98c46a71b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:55.005063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263b348-8101-4955-bf80-32071b12e13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:55.005700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b348-8101-4955-bf80-32071b12e13b service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:55.005987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b348-8101-4955-bf80-32071b12e13b service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:55.006269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b348-8101-4955-bf80-32071b12e13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:55.014573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263b348-8101-4955-bf80-32071b12e13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:55.016201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263b348-8101-4955-bf80-32071b12e13b service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:55.026233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 720/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:55.947419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bdc4c-c9af-4482-ae18-3b2280e350ee None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:55.947976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bdc4c-c9af-4482-ae18-3b2280e350ee None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:55.947976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bdc4c-c9af-4482-ae18-3b2280e350ee None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:55.949381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:55.954838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768322e-faff-41c0-a84f-e54e6b14d597 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:55.955379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768322e-faff-41c0-a84f-e54e6b14d597 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:55.955552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768322e-faff-41c0-a84f-e54e6b14d597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:55.975214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a768322e-faff-41c0-a84f-e54e6b14d597 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:55.978684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a768322e-faff-41c0-a84f-e54e6b14d597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:55.984291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:56.133186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bdc4c-c9af-4482-ae18-3b2280e350ee None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:56.147247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bdc4c-c9af-4482-ae18-3b2280e350ee None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:56.152588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b2809-1e28-48a8-84b8-61e8529d8420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:56.153220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b2809-1e28-48a8-84b8-61e8529d8420 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:56.153505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b2809-1e28-48a8-84b8-61e8529d8420 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:56.153784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b2809-1e28-48a8-84b8-61e8529d8420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:56.162758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b2809-1e28-48a8-84b8-61e8529d8420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:56.164372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b2809-1e28-48a8-84b8-61e8529d8420 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:56.173697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 716/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:57.086199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c974dd-44c3-4012-bcc0-a5a8f4397981 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:57.086774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c974dd-44c3-4012-bcc0-a5a8f4397981 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:57.086774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c974dd-44c3-4012-bcc0-a5a8f4397981 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:57.088172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:57.093332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8bc68-a191-48ce-9ed6-5115821ac568 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:57.093791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8bc68-a191-48ce-9ed6-5115821ac568 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:57.093910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8bc68-a191-48ce-9ed6-5115821ac568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:57.111859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e4a8bc68-a191-48ce-9ed6-5115821ac568 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:57.113580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e4a8bc68-a191-48ce-9ed6-5115821ac568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:57.121513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:57.270882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c974dd-44c3-4012-bcc0-a5a8f4397981 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:57.285187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c974dd-44c3-4012-bcc0-a5a8f4397981 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:57.290815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d75ef-13cc-4b65-9130-9b6a671fd9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:57.291497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d75ef-13cc-4b65-9130-9b6a671fd9e1 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:57.291793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d75ef-13cc-4b65-9130-9b6a671fd9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:57.292076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d75ef-13cc-4b65-9130-9b6a671fd9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:57.301069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d75ef-13cc-4b65-9130-9b6a671fd9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:57.302673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d75ef-13cc-4b65-9130-9b6a671fd9e1 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:57.312546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 723/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:20:59.449245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468768e9-a128-4400-a593-275369dbbf7a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:59.449827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468768e9-a128-4400-a593-275369dbbf7a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:59.449827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468768e9-a128-4400-a593-275369dbbf7a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:59.451156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:20:59.456583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18010698-8871-4b0a-b0b4-922f63a6d568 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:59.457525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18010698-8871-4b0a-b0b4-922f63a6d568 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:59.457736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18010698-8871-4b0a-b0b4-922f63a6d568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:59.475925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-18010698-8871-4b0a-b0b4-922f63a6d568 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:20:59.477111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-18010698-8871-4b0a-b0b4-922f63a6d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:20:59.484887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:20:59.523720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468768e9-a128-4400-a593-275369dbbf7a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:20:59.534296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468768e9-a128-4400-a593-275369dbbf7a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:20:59.538332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc9e4-240a-49c6-9092-fbe9ef91051b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:20:59.540952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc9e4-240a-49c6-9092-fbe9ef91051b service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:20:59.540952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc9e4-240a-49c6-9092-fbe9ef91051b service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:20:59.540952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc9e4-240a-49c6-9092-fbe9ef91051b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:20:59.548378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc9e4-240a-49c6-9092-fbe9ef91051b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:20:59.549806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc9e4-240a-49c6-9092-fbe9ef91051b service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:20:59.559162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 719/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:02.157144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96deb3-28fa-4ca3-a34f-92c6388f4896 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:02.157714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96deb3-28fa-4ca3-a34f-92c6388f4896 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:02.157714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96deb3-28fa-4ca3-a34f-92c6388f4896 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:02.159057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:02.165083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cb3c2-66cb-4c7a-b90a-d70d2769ee4e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:02.165535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cb3c2-66cb-4c7a-b90a-d70d2769ee4e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:02.165642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cb3c2-66cb-4c7a-b90a-d70d2769ee4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:02.183803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fb9cb3c2-66cb-4c7a-b90a-d70d2769ee4e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:02.184993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fb9cb3c2-66cb-4c7a-b90a-d70d2769ee4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:02.192871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:21:02.231435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96deb3-28fa-4ca3-a34f-92c6388f4896 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:02.243243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96deb3-28fa-4ca3-a34f-92c6388f4896 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:02.246921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db9eb-88d3-4dc6-929d-b1861ff51c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:02.247591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db9eb-88d3-4dc6-929d-b1861ff51c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:02.247910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db9eb-88d3-4dc6-929d-b1861ff51c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:02.248192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db9eb-88d3-4dc6-929d-b1861ff51c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:02.257521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db9eb-88d3-4dc6-929d-b1861ff51c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:02.258952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db9eb-88d3-4dc6-929d-b1861ff51c4b service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:02.268260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 726/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:05.018785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59fa5-b2a0-49a6-ba16-fbd3be28241d None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:05.020035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59fa5-b2a0-49a6-ba16-fbd3be28241d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:05.020035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb59fa5-b2a0-49a6-ba16-fbd3be28241d None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:05.021393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:21:05.026880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511ef5-acbe-4fb9-a1c1-18660b5bb95f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:05.027294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511ef5-acbe-4fb9-a1c1-18660b5bb95f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:05.027480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511ef5-acbe-4fb9-a1c1-18660b5bb95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:05.045357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-79511ef5-acbe-4fb9-a1c1-18660b5bb95f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:05.046549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-79511ef5-acbe-4fb9-a1c1-18660b5bb95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:05.054687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:05.093056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb59fa5-b2a0-49a6-ba16-fbd3be28241d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:05.104132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb59fa5-b2a0-49a6-ba16-fbd3be28241d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:05.107635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8eada-3e7d-485f-82ed-7ad746491e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:05.108264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8eada-3e7d-485f-82ed-7ad746491e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:05.108547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8eada-3e7d-485f-82ed-7ad746491e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:05.108825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8eada-3e7d-485f-82ed-7ad746491e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:05.118464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8eada-3e7d-485f-82ed-7ad746491e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:05.119924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8eada-3e7d-485f-82ed-7ad746491e14 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:05.129178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 722/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:07.500823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790324-c5ab-4124-be54-fda8da7898bc None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:07.501398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790324-c5ab-4124-be54-fda8da7898bc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:07.501398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790324-c5ab-4124-be54-fda8da7898bc None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:07.502802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:07.508295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618364-c977-440a-aec3-e9cd75da956b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:07.508722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618364-c977-440a-aec3-e9cd75da956b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:07.508873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618364-c977-440a-aec3-e9cd75da956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:07.526758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5b618364-c977-440a-aec3-e9cd75da956b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:07.527984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5b618364-c977-440a-aec3-e9cd75da956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:07.536437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:07.574499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84790324-c5ab-4124-be54-fda8da7898bc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:07.586345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84790324-c5ab-4124-be54-fda8da7898bc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:07.590054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793a7cb-500f-44f7-967b-c4bf1d0abc74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:07.590729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793a7cb-500f-44f7-967b-c4bf1d0abc74 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:07.591012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793a7cb-500f-44f7-967b-c4bf1d0abc74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:07.591288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793a7cb-500f-44f7-967b-c4bf1d0abc74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:07.601932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793a7cb-500f-44f7-967b-c4bf1d0abc74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:07.603454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793a7cb-500f-44f7-967b-c4bf1d0abc74 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:07.613569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 729/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:09.935989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4329645-efe2-459c-8598-4ff3173cc7b1 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:09.936519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4329645-efe2-459c-8598-4ff3173cc7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:09.936519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4329645-efe2-459c-8598-4ff3173cc7b1 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:09.937794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:09.943817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2efab-d765-4f12-af00-e2074038309f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:09.944264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2efab-d765-4f12-af00-e2074038309f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:09.944391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2efab-d765-4f12-af00-e2074038309f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:09.962150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-70f2efab-d765-4f12-af00-e2074038309f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:09.963373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-70f2efab-d765-4f12-af00-e2074038309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:09.971093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:10.010022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4329645-efe2-459c-8598-4ff3173cc7b1 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:10.021805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4329645-efe2-459c-8598-4ff3173cc7b1 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:10.026585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99074bb-e649-454f-abbc-57d37ee7133b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:10.027223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99074bb-e649-454f-abbc-57d37ee7133b service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:10.027540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99074bb-e649-454f-abbc-57d37ee7133b service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:10.027830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99074bb-e649-454f-abbc-57d37ee7133b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:10.037369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99074bb-e649-454f-abbc-57d37ee7133b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:10.038807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99074bb-e649-454f-abbc-57d37ee7133b service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:10.048196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 725/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:12.338318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5962-1bf1-464a-ba0d-c9eb4377c5f6 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:12.338882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5962-1bf1-464a-ba0d-c9eb4377c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:12.338882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5962-1bf1-464a-ba0d-c9eb4377c5f6 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:12.340400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:12.345785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2e076-94f9-43a8-9e9d-82e785d7f38f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:12.346197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2e076-94f9-43a8-9e9d-82e785d7f38f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:12.346339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2e076-94f9-43a8-9e9d-82e785d7f38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:12.365496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d4b2e076-94f9-43a8-9e9d-82e785d7f38f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:12.366732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d4b2e076-94f9-43a8-9e9d-82e785d7f38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:12.374934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:21:12.413494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac5962-1bf1-464a-ba0d-c9eb4377c5f6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:12.424278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac5962-1bf1-464a-ba0d-c9eb4377c5f6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:12.428591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20441365-91d2-4dda-bd68-fe96cb34949c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:12.429192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20441365-91d2-4dda-bd68-fe96cb34949c service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:12.429472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20441365-91d2-4dda-bd68-fe96cb34949c service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:12.429744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20441365-91d2-4dda-bd68-fe96cb34949c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:12.439506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20441365-91d2-4dda-bd68-fe96cb34949c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:12.441054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20441365-91d2-4dda-bd68-fe96cb34949c service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:12.451677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 732/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:14.775328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ea43e-68d1-4447-a213-4fcc4ac37016 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:14.775930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ea43e-68d1-4447-a213-4fcc4ac37016 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:14.775930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ea43e-68d1-4447-a213-4fcc4ac37016 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:14.777311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:14.782798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b34e7f-6879-4fb2-a053-38349d0ee7d5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:14.783266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b34e7f-6879-4fb2-a053-38349d0ee7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:14.783388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b34e7f-6879-4fb2-a053-38349d0ee7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:14.789429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=70 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:21:14.801951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-37b34e7f-6879-4fb2-a053-38349d0ee7d5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:14.803166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-37b34e7f-6879-4fb2-a053-38349d0ee7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:14.811207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:14.851290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ea43e-68d1-4447-a213-4fcc4ac37016 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:14.856695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=73 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:21:14.862692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ea43e-68d1-4447-a213-4fcc4ac37016 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:14.868234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faedd49-da41-4934-a677-08afc0c8a2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:14.868234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faedd49-da41-4934-a677-08afc0c8a2eb service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:14.868234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faedd49-da41-4934-a677-08afc0c8a2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:14.868234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faedd49-da41-4934-a677-08afc0c8a2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:14.877635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faedd49-da41-4934-a677-08afc0c8a2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:14.879717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faedd49-da41-4934-a677-08afc0c8a2eb service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:14.889544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 728/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:17.187549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64adbbe-6a0a-4678-9eb3-d82b45c67d9f None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:17.188076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64adbbe-6a0a-4678-9eb3-d82b45c67d9f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:17.188076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64adbbe-6a0a-4678-9eb3-d82b45c67d9f None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:17.189387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:17.195469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700929d-f9be-47cc-a6be-21479650db10 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:17.195923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700929d-f9be-47cc-a6be-21479650db10 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:17.196035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700929d-f9be-47cc-a6be-21479650db10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:17.213843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a700929d-f9be-47cc-a6be-21479650db10 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:17.215018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a700929d-f9be-47cc-a6be-21479650db10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:17.222852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:17.259835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64adbbe-6a0a-4678-9eb3-d82b45c67d9f None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:17.271512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64adbbe-6a0a-4678-9eb3-d82b45c67d9f None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:17.274963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a257bd-1ea7-45cb-9387-22966a3b83e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:17.275616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a257bd-1ea7-45cb-9387-22966a3b83e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:17.275899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a257bd-1ea7-45cb-9387-22966a3b83e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:17.276208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a257bd-1ea7-45cb-9387-22966a3b83e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:17.285453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a257bd-1ea7-45cb-9387-22966a3b83e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:17.286749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a257bd-1ea7-45cb-9387-22966a3b83e8 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:17.296499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 735/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:19.533863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c65e55-bdbf-4c9b-babc-4a918ec58422 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:19.535033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c65e55-bdbf-4c9b-babc-4a918ec58422 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:19.535033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c65e55-bdbf-4c9b-babc-4a918ec58422 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:19.536409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:19.541851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7369d0-c362-41c2-b5e9-d5e792928e8c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:19.542285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7369d0-c362-41c2-b5e9-d5e792928e8c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:19.547374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7369d0-c362-41c2-b5e9-d5e792928e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:19.560079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-eb7369d0-c362-41c2-b5e9-d5e792928e8c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:19.561262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-eb7369d0-c362-41c2-b5e9-d5e792928e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:19.569580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:19.606780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c65e55-bdbf-4c9b-babc-4a918ec58422 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:19.618089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c65e55-bdbf-4c9b-babc-4a918ec58422 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:19.621672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ed9dd-a558-4b17-bd94-f91710455625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:19.622314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed9dd-a558-4b17-bd94-f91710455625 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:19.622599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed9dd-a558-4b17-bd94-f91710455625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:19.622881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ed9dd-a558-4b17-bd94-f91710455625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:19.640278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ed9dd-a558-4b17-bd94-f91710455625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:19.640278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ed9dd-a558-4b17-bd94-f91710455625 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:19.643457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 731/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:21.865551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e51e-a64f-46f5-82eb-4a1a598c942b None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:21.866190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e51e-a64f-46f5-82eb-4a1a598c942b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:21.866319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e51e-a64f-46f5-82eb-4a1a598c942b None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:21.868004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:21.873548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365137-c0d9-4696-995d-25f9faa43cb6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:21.873977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365137-c0d9-4696-995d-25f9faa43cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:21.874163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28365137-c0d9-4696-995d-25f9faa43cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:21.892326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-28365137-c0d9-4696-995d-25f9faa43cb6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:21.894075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-28365137-c0d9-4696-995d-25f9faa43cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:21.902029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:21.939331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e51e-a64f-46f5-82eb-4a1a598c942b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:21.950211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e51e-a64f-46f5-82eb-4a1a598c942b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:21.953700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721e4e3-dfe6-4c57-be79-714f953787ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:21.954328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721e4e3-dfe6-4c57-be79-714f953787ac service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:21.954612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721e4e3-dfe6-4c57-be79-714f953787ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:21.954887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721e4e3-dfe6-4c57-be79-714f953787ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:21.964135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721e4e3-dfe6-4c57-be79-714f953787ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:21.965571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721e4e3-dfe6-4c57-be79-714f953787ac service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:21.976058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 738/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:31.886654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=17 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:21:31.970658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=17 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:21:54.065135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3cdf6-aeac-427b-8e3b-11cf5643a4bd None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:54.065727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3cdf6-aeac-427b-8e3b-11cf5643a4bd None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:54.065727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3cdf6-aeac-427b-8e3b-11cf5643a4bd None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:54.067097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 733/1471] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:54.072316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403e951-9397-421a-9e4c-df38d2476f9d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:54.072776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403e951-9397-421a-9e4c-df38d2476f9d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:54.072933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403e951-9397-421a-9e4c-df38d2476f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:54.091098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d403e951-9397-421a-9e4c-df38d2476f9d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:54.092329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d403e951-9397-421a-9e4c-df38d2476f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:54.100319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 739/1472] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:54.147583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e615fc-bf31-40b1-9aad-42400fd116bb None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:54.147873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e615fc-bf31-40b1-9aad-42400fd116bb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:54.148153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e615fc-bf31-40b1-9aad-42400fd116bb None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:54.149506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 734/1473] 10.222.0.18 () {54 vars in 962 bytes} [Wed Jun 1 04:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:54.154290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88507da5-1c09-430f-a274-841684414851 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:54.154590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88507da5-1c09-430f-a274-841684414851 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:54.154875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88507da5-1c09-430f-a274-841684414851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:54.172878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-88507da5-1c09-430f-a274-841684414851 None None] MFA Rules not processed for user `2e0959756bd54aa58afb14b9b45d29ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:54.174590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-88507da5-1c09-430f-a274-841684414851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:54.181182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 740/1474] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Jun 1 04:21:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:54.188663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e615fc-bf31-40b1-9aad-42400fd116bb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:54.200002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e615fc-bf31-40b1-9aad-42400fd116bb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:54.203487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f567062-c659-4270-8d8c-937a3dd4ebf2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:54.204128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f567062-c659-4270-8d8c-937a3dd4ebf2 service glance] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:54.204456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f567062-c659-4270-8d8c-937a3dd4ebf2 service glance] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:54.204673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f567062-c659-4270-8d8c-937a3dd4ebf2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:54.213437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f567062-c659-4270-8d8c-937a3dd4ebf2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:54.214865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f567062-c659-4270-8d8c-937a3dd4ebf2 service glance] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:54.224262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 735/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:21:56.567437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef57a40-5ddc-4ec4-bd14-facbf9e88455 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:56.568006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef57a40-5ddc-4ec4-bd14-facbf9e88455 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:56.568006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef57a40-5ddc-4ec4-bd14-facbf9e88455 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:56.569342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 741/1476] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:56.574633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d7919-9e88-4078-8689-760fa836ba46 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:56.575077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d7919-9e88-4078-8689-760fa836ba46 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:56.575192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d7919-9e88-4078-8689-760fa836ba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:56.593245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c80d7919-9e88-4078-8689-760fa836ba46 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:56.594427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c80d7919-9e88-4078-8689-760fa836ba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:56.602333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 736/1477] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:56.626441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef57a40-5ddc-4ec4-bd14-facbf9e88455 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:56.640054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef57a40-5ddc-4ec4-bd14-facbf9e88455 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:56.644495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:56.645132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:56.645411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:56.645682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:56.646407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:56.648181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:21:56.648181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:21:56.648181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:21:56.649332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37964d80-c172-4f04-ba43-e15601320f97 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:56.668564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 742/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jun 1 04:21:56 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:21:58.953566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadad718-5403-44f1-b649-5be808394fd3 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:58.954109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadad718-5403-44f1-b649-5be808394fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:58.954109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadad718-5403-44f1-b649-5be808394fd3 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:58.956074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 737/1479] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:21:58.961543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e741b-3d8f-4fe1-84f2-bab58cb9c592 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:58.961964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e741b-3d8f-4fe1-84f2-bab58cb9c592 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:58.962103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e741b-3d8f-4fe1-84f2-bab58cb9c592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:58.979749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d16e741b-3d8f-4fe1-84f2-bab58cb9c592 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:21:58.980937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d16e741b-3d8f-4fe1-84f2-bab58cb9c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:21:58.988701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 743/1480] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:21:59.033003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadad718-5403-44f1-b649-5be808394fd3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:21:59.047161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadad718-5403-44f1-b649-5be808394fd3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:21:59.052795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17991a27-c4e7-4a46-bfc0-d58172b7f625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:21:59.053432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991a27-c4e7-4a46-bfc0-d58172b7f625 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:21:59.053720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991a27-c4e7-4a46-bfc0-d58172b7f625 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:21:59.054002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991a27-c4e7-4a46-bfc0-d58172b7f625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:21:59.062440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17991a27-c4e7-4a46-bfc0-d58172b7f625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:21:59.064671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17991a27-c4e7-4a46-bfc0-d58172b7f625 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:21:59.074434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 738/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:22:01.567970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1be3a-6116-4763-abb9-87548789341c None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:01.568569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1be3a-6116-4763-abb9-87548789341c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:01.568569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1be3a-6116-4763-abb9-87548789341c None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:01.569987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 744/1482] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:22:01.575539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f8eb-79ae-48fd-87ec-37d081cf6a2f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:01.575972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f8eb-79ae-48fd-87ec-37d081cf6a2f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:01.576184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f8eb-79ae-48fd-87ec-37d081cf6a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:01.594340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3767f8eb-79ae-48fd-87ec-37d081cf6a2f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:22:01.595722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3767f8eb-79ae-48fd-87ec-37d081cf6a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:22:01.604181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 739/1483] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:22:01.640128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1be3a-6116-4763-abb9-87548789341c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:22:01.651499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1be3a-6116-4763-abb9-87548789341c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:22:01.656066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cdc14-055e-47a8-8b37-3c6a5571ec71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:22:01.656707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cdc14-055e-47a8-8b37-3c6a5571ec71 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:01.656991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cdc14-055e-47a8-8b37-3c6a5571ec71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:01.657333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cdc14-055e-47a8-8b37-3c6a5571ec71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:01.667006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cdc14-055e-47a8-8b37-3c6a5571ec71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:22:01.668525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cdc14-055e-47a8-8b37-3c6a5571ec71 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:22:01.677941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 745/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:22:03.973310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e0c4-a2f9-4b17-9824-172f116351b5 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:03.973820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e0c4-a2f9-4b17-9824-172f116351b5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:03.973820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e0c4-a2f9-4b17-9824-172f116351b5 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:03.975163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 740/1485] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:22:03.980446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777e39-2988-40c3-9be4-fd2b8654cf76 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:03.980909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777e39-2988-40c3-9be4-fd2b8654cf76 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:03.981014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777e39-2988-40c3-9be4-fd2b8654cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:03.999224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c0777e39-2988-40c3-9be4-fd2b8654cf76 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:22:04.000451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c0777e39-2988-40c3-9be4-fd2b8654cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:22:04.008942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 746/1486] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:22:04.046983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e0c4-a2f9-4b17-9824-172f116351b5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:22:04.058063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e0c4-a2f9-4b17-9824-172f116351b5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:22:04.061579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad624a-7c3d-48c5-9c6f-5f64a9d75fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:22:04.062212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad624a-7c3d-48c5-9c6f-5f64a9d75fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:04.062497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad624a-7c3d-48c5-9c6f-5f64a9d75fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:04.062775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad624a-7c3d-48c5-9c6f-5f64a9d75fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:04.072121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad624a-7c3d-48c5-9c6f-5f64a9d75fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:22:04.073547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad624a-7c3d-48c5-9c6f-5f64a9d75fc9 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:22:04.082833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 741/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:22:06.485119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52d22b-8586-4400-ba60-c623153fee3c None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:06.485726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52d22b-8586-4400-ba60-c623153fee3c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:06.485726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52d22b-8586-4400-ba60-c623153fee3c None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:06.487042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 747/1488] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:22:06.492537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503ee1-4a3c-4c48-8747-e3ad1906a6b6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:06.492976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503ee1-4a3c-4c48-8747-e3ad1906a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:06.493103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503ee1-4a3c-4c48-8747-e3ad1906a6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:06.511160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-59503ee1-4a3c-4c48-8747-e3ad1906a6b6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:22:06.512389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-59503ee1-4a3c-4c48-8747-e3ad1906a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:22:06.520719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 742/1489] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:22:06.560093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52d22b-8586-4400-ba60-c623153fee3c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:22:06.571478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52d22b-8586-4400-ba60-c623153fee3c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:22:06.574998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8ee81-5de5-4376-acdd-8009a1e51195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:22:06.575668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ee81-5de5-4376-acdd-8009a1e51195 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:06.575955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ee81-5de5-4376-acdd-8009a1e51195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:06.576234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ee81-5de5-4376-acdd-8009a1e51195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:06.586136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8ee81-5de5-4376-acdd-8009a1e51195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:22:06.587603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8ee81-5de5-4376-acdd-8009a1e51195 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:22:06.597001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 748/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:22:09.318594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf287-093b-4b80-8152-53e342c2ae87 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:09.319174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf287-093b-4b80-8152-53e342c2ae87 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:09.319327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf287-093b-4b80-8152-53e342c2ae87 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:09.320610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 743/1491] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jun 1 04:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:22:09.325969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40755bb-28cb-4b22-977c-d1b519b8b7c3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:09.326417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40755bb-28cb-4b22-977c-d1b519b8b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:09.326514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40755bb-28cb-4b22-977c-d1b519b8b7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:09.344564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f40755bb-28cb-4b22-977c-d1b519b8b7c3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:22:09.346292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f40755bb-28cb-4b22-977c-d1b519b8b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:22:09.354073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 749/1492] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jun 1 04:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:22:09.394092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daf287-093b-4b80-8152-53e342c2ae87 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:22:09.404839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daf287-093b-4b80-8152-53e342c2ae87 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:22:09.408455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e049b4-c819-49df-a37b-1e05127db905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:22:09.409094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e049b4-c819-49df-a37b-1e05127db905 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:22:09.409378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e049b4-c819-49df-a37b-1e05127db905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:22:09.409656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e049b4-c819-49df-a37b-1e05127db905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:22:09.418582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e049b4-c819-49df-a37b-1e05127db905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:22:09.420045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e049b4-c819-49df-a37b-1e05127db905 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:22:09.429376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 744/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:22:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:22:19.339416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=33 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:22:19.425237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=33 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:23:43.939001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:43.939601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:43.939601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:43.958078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:23:43.959289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:43.967158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 750/1494] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:23:43.975534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:43.976082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:43.976082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:43.996920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:23:43.998128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:44.006911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 745/1495] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:23:44.011606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.021992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5940e-13a9-4d08-9287-56d3d664a15d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.026643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.027397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.028068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.028068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.028593 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:23:44.028906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.030725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.030725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:23:44.030725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:23:44.032244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.044092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 751/1496] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:23:44.052412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.061805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769bd87-e789-4e0e-9f55-089f85dd21cb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.066527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.067181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.067506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.067782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.068505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.070241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.070241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:23:44.070241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:23:44.071487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.185005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 746/1497] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:23:44 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:23:44.189554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.199569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19049089-7152-4a51-a6c1-86ab95016381 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.259503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.260182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.260466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.260749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.261450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.262600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.262600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:23:44.262600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:23:44.263718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.373582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 752/1498] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:23:44 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:23:44.378212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.388602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765d872-8167-4382-935c-e41e82357835 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.393713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.394400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.394731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.395006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.395874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.396787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.396787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:23:44.396787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:23:44.397659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.410306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 747/1499] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:23:44.414279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.424084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82542baf-3e56-43a0-97a9-006bd2c8f963 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.428727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.429373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.429656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.429933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.430647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.431715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.431715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:23:44.431715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:23:44.432586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c3e5e-0a46-4b46-85d1-78d14961a0b7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.446340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 753/1500] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:23:44.450309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.461256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8176a1d-5c8c-41c0-8a7e-f113a98b2f14 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.466033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.466912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.467198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.467506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] PATH_INFO: `/v3/projects/1e02e0004424489882567185494f4919/users/fe00f700ded347f5a598c6c21b47f1f0/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.482230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e02e0004424489882567185494f4919, user_id=fe00f700ded347f5a598c6c21b47f1f0, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.483263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.483263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:23:44.483263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:23:44.488722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd1d7-6c8e-44f7-aa26-d6b1a16d5a0b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.619401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 748/1501] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:23:44 2022] PUT /identity/v3/projects/1e02e0004424489882567185494f4919/users/fe00f700ded347f5a598c6c21b47f1f0/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:23:44.632541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f9a58-4009-4088-be97-59ea04c2ec9c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.632855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f9a58-4009-4088-be97-59ea04c2ec9c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.633126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f9a58-4009-4088-be97-59ea04c2ec9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.661692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9e6f9a58-4009-4088-be97-59ea04c2ec9c None None] MFA Rules not processed for user `fe00f700ded347f5a598c6c21b47f1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:23:44.664711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9e6f9a58-4009-4088-be97-59ea04c2ec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:44.728712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 754/1502] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:23:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:23:44.742406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c41f0-375c-460a-8d38-8c4a81607534 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.742704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c41f0-375c-460a-8d38-8c4a81607534 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.742987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c41f0-375c-460a-8d38-8c4a81607534 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.744430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 749/1503] 10.222.0.18 () {54 vars in 963 bytes} [Wed Jun 1 04:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:23:44.749568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e234d1-7fdb-4a1e-a423-7ba1b542343e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.749971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e234d1-7fdb-4a1e-a423-7ba1b542343e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.750153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e234d1-7fdb-4a1e-a423-7ba1b542343e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.768385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-41e234d1-7fdb-4a1e-a423-7ba1b542343e None None] MFA Rules not processed for user `3f4e0e17040d41bf91cf4022dc4b4d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:23:44.769594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-41e234d1-7fdb-4a1e-a423-7ba1b542343e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:44.814501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:23:44.821727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 755/1504] 10.222.0.18 () {58 vars in 1060 bytes} [Wed Jun 1 04:23:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:23:44.829895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c41f0-375c-460a-8d38-8c4a81607534 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.840785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c41f0-375c-460a-8d38-8c4a81607534 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.846417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.846417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.846417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.846417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.854272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.855734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:44.864681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 750/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:23:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:23:44.909068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44af4-df52-4ad9-a211-6ce979571961 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.909356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44af4-df52-4ad9-a211-6ce979571961 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.909636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44af4-df52-4ad9-a211-6ce979571961 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.910917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 756/1506] 10.222.0.18 () {52 vars in 829 bytes} [Wed Jun 1 04:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:23:44.914482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:44.976227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd610b4-d138-41b8-a9fc-c48ca906ae0d service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:44.980936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:44.981817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:44.982105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:44.982381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] PATH_INFO: `/v3/users/fe00f700ded347f5a598c6c21b47f1f0` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:44.983606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe00f700ded347f5a598c6c21b47f1f0)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:44.984702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:44.984702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:23:44.984702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:23:44.985908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:45.195656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe00f700ded347f5a598c6c21b47f1f0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:45.207796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe00f700ded347f5a598c6c21b47f1f0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:45.208450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe00f700ded347f5a598c6c21b47f1f0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:45.227073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe00f700ded347f5a598c6c21b47f1f0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:45.230938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:23:45.326789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe00f700ded347f5a598c6c21b47f1f0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:45.333432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 751/1507] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:23:44 2022] DELETE /identity/v3/users/fe00f700ded347f5a598c6c21b47f1f0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:23:45.337923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:45.338458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:45.338458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:45.356665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:23:45.357848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:45.416262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 757/1508] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:23:45.425766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:45.428841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:45.490591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fe687-387d-448f-a5fc-8aab1a42e3c1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:45.490591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632c973-e7ed-44f3-ace6-56e4ad78ccae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:45.490591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632c973-e7ed-44f3-ace6-56e4ad78ccae service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:45.490988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632c973-e7ed-44f3-ace6-56e4ad78ccae service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:45.490988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632c973-e7ed-44f3-ace6-56e4ad78ccae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:45.500599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632c973-e7ed-44f3-ace6-56e4ad78ccae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:45.502074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632c973-e7ed-44f3-ace6-56e4ad78ccae service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:45.511642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 752/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:23:45.957871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:23:45.959868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:23:45.973311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbc788-11df-47b6-a39e-35fd2e1006e8 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:23:45.978195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:23:45.979210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:23:45.980110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:23:45.980110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] PATH_INFO: `/v3/projects/1e02e0004424489882567185494f4919` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:23:45.981353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e02e0004424489882567185494f4919)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:23:45.982470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:23:45.982470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:23:45.982470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:23:45.983762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:23:46.101655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e02e0004424489882567185494f4919', 'request_id': 'req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:46.107602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e02e0004424489882567185494f4919', 'request_id': 'req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:46.187652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e02e0004424489882567185494f4919', 'request_id': 'req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:46.206148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] The token cache is being invalidate because project 1e02e0004424489882567185494f4919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:23:46.206438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:46.207162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ad3f312b-cb31-46d8-9f88-bf57a15b62c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:23:46.208738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 758/1510] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:23:45 2022] DELETE /identity/v3/projects/1e02e0004424489882567185494f4919 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:23:55.507488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:23:56.202630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:25:23.940513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff34a6-5d27-46d1-b08f-44366c39ef1a None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:23.941077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff34a6-5d27-46d1-b08f-44366c39ef1a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:23.941077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff34a6-5d27-46d1-b08f-44366c39ef1a None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:23.942462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:23.948180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ecdc9-419c-4caf-b90e-d7b7f8457621 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:23.948666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ecdc9-419c-4caf-b90e-d7b7f8457621 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:23.948825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ecdc9-419c-4caf-b90e-d7b7f8457621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:23.968202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b37ecdc9-419c-4caf-b90e-d7b7f8457621 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:23.969416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b37ecdc9-419c-4caf-b90e-d7b7f8457621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:24.029931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:24.200027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff34a6-5d27-46d1-b08f-44366c39ef1a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:24.201826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4dff34a6-5d27-46d1-b08f-44366c39ef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:24.279229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff34a6-5d27-46d1-b08f-44366c39ef1a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:24.284686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f080b5-bba8-4d1e-b8e4-fbd6c22bba62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:24.285345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f080b5-bba8-4d1e-b8e4-fbd6c22bba62 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:24.285629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f080b5-bba8-4d1e-b8e4-fbd6c22bba62 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:24.285910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f080b5-bba8-4d1e-b8e4-fbd6c22bba62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:24.295004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f080b5-bba8-4d1e-b8e4-fbd6c22bba62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:24.296624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f080b5-bba8-4d1e-b8e4-fbd6c22bba62 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:24.306179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 754/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:25.176152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab91b37-bc88-4cd8-9575-c06dbb6fb144 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:25.176706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab91b37-bc88-4cd8-9575-c06dbb6fb144 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:25.176706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab91b37-bc88-4cd8-9575-c06dbb6fb144 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:25.178086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:25.183287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954be2c-a192-42c8-a5d2-f6ac1627404a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:25.183697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954be2c-a192-42c8-a5d2-f6ac1627404a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:25.183897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954be2c-a192-42c8-a5d2-f6ac1627404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:25.202100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a954be2c-a192-42c8-a5d2-f6ac1627404a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:25.203292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a954be2c-a192-42c8-a5d2-f6ac1627404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:25.211021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:25.362802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab91b37-bc88-4cd8-9575-c06dbb6fb144 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:25.377478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab91b37-bc88-4cd8-9575-c06dbb6fb144 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:25.383089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f91be-2b2a-4593-8e1a-ae3e4951daef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:25.383743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f91be-2b2a-4593-8e1a-ae3e4951daef service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:25.384030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f91be-2b2a-4593-8e1a-ae3e4951daef service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:25.384306 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f91be-2b2a-4593-8e1a-ae3e4951daef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:25.392556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f91be-2b2a-4593-8e1a-ae3e4951daef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:25.394123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f91be-2b2a-4593-8e1a-ae3e4951daef service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:25.403412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 761/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:25.549346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285643e0-c834-48a5-8eb8-378e4d72523e None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:25.549640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285643e0-c834-48a5-8eb8-378e4d72523e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:25.549927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285643e0-c834-48a5-8eb8-378e4d72523e None None] PATH_INFO: `/v3` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:25.551274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 756/1517] 10.222.0.45 () {54 vars in 925 bytes} [Wed Jun 1 04:25:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:25:25.557797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb1a4-ad5a-4672-8f19-97a443ade1c1 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:25.558219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb1a4-ad5a-4672-8f19-97a443ade1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:25.558510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb1a4-ad5a-4672-8f19-97a443ade1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:25.576510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-303cb1a4-ad5a-4672-8f19-97a443ade1c1 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:25.577718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-303cb1a4-ad5a-4672-8f19-97a443ade1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:25.640165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 762/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Jun 1 04:25:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:34.988372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6a8-73eb-4ae5-af97-4822ae7ccb1c None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:34.988923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6a8-73eb-4ae5-af97-4822ae7ccb1c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:34.988923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6a8-73eb-4ae5-af97-4822ae7ccb1c None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:34.990291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 757/1519] 10.222.0.18 () {54 vars in 960 bytes} [Wed Jun 1 04:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:34.996100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c41f2-1eb4-4396-8c02-7291d9d02723 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:34.996532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c41f2-1eb4-4396-8c02-7291d9d02723 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:34.996659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c41f2-1eb4-4396-8c02-7291d9d02723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:35.014343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-919c41f2-1eb4-4396-8c02-7291d9d02723 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:35.015545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-919c41f2-1eb4-4396-8c02-7291d9d02723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:35.023809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 763/1520] 10.222.0.18 () {58 vars in 1057 bytes} [Wed Jun 1 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:35.031368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b6a8-73eb-4ae5-af97-4822ae7ccb1c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:35.045419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b6a8-73eb-4ae5-af97-4822ae7ccb1c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:35.050551 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa55e4-daf3-4898-8cb8-b710310b2eae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:35.051189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa55e4-daf3-4898-8cb8-b710310b2eae service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:35.051505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa55e4-daf3-4898-8cb8-b710310b2eae service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:35.051793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa55e4-daf3-4898-8cb8-b710310b2eae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:35.060862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa55e4-daf3-4898-8cb8-b710310b2eae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:35.062435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa55e4-daf3-4898-8cb8-b710310b2eae service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:35.071864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 758/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:36.696301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347b5b-ba47-4834-add8-1c53b9274916 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:36.696832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347b5b-ba47-4834-add8-1c53b9274916 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:36.696947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347b5b-ba47-4834-add8-1c53b9274916 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:36.698258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:36.703070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f06f08-785e-4c5e-a290-660713a64026 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:36.703516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f06f08-785e-4c5e-a290-660713a64026 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:36.703682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f06f08-785e-4c5e-a290-660713a64026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:36.721369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-13f06f08-785e-4c5e-a290-660713a64026 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:36.722555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-13f06f08-785e-4c5e-a290-660713a64026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:36.730851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:36.884701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d347b5b-ba47-4834-add8-1c53b9274916 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:36.899652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d347b5b-ba47-4834-add8-1c53b9274916 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:36.905036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14512f7-6750-461b-8c8e-4e6fba6c94d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:36.905666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14512f7-6750-461b-8c8e-4e6fba6c94d4 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:36.905953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14512f7-6750-461b-8c8e-4e6fba6c94d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:36.906233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14512f7-6750-461b-8c8e-4e6fba6c94d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:36.915077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14512f7-6750-461b-8c8e-4e6fba6c94d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:36.922096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14512f7-6750-461b-8c8e-4e6fba6c94d4 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:36.926349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 765/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:37.795000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747d847-f113-4483-a453-76b949f28b55 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:37.795582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747d847-f113-4483-a453-76b949f28b55 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:37.795698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747d847-f113-4483-a453-76b949f28b55 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:37.796985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 1 04:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:37.801830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519900d9-ae68-447b-a588-37d14263263f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:37.802259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519900d9-ae68-447b-a588-37d14263263f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:37.802393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519900d9-ae68-447b-a588-37d14263263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:37.821725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-519900d9-ae68-447b-a588-37d14263263f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:37.822916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-519900d9-ae68-447b-a588-37d14263263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:37.830598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 1 04:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:37.979930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747d847-f113-4483-a453-76b949f28b55 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:37.994171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747d847-f113-4483-a453-76b949f28b55 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:37.999516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c74faf-c9ac-4df2-b00c-49bf235188a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:38.000218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c74faf-c9ac-4df2-b00c-49bf235188a0 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:38.000443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c74faf-c9ac-4df2-b00c-49bf235188a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:38.000730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c74faf-c9ac-4df2-b00c-49bf235188a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:38.009943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c74faf-c9ac-4df2-b00c-49bf235188a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:38.011557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c74faf-c9ac-4df2-b00c-49bf235188a0 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:38.020837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 761/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:42.102052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a2e02-ed13-4f56-b1c1-771365e08b3a None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:42.102649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a2e02-ed13-4f56-b1c1-771365e08b3a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:42.102649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a2e02-ed13-4f56-b1c1-771365e08b3a None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:42.104022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:42.109411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c478ed-0d85-44de-b93a-b339dbb77d40 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:42.109870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c478ed-0d85-44de-b93a-b339dbb77d40 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:42.109979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c478ed-0d85-44de-b93a-b339dbb77d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:42.128716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b5c478ed-0d85-44de-b93a-b339dbb77d40 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:42.129906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b5c478ed-0d85-44de-b93a-b339dbb77d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:42.138216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:42.173950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a2e02-ed13-4f56-b1c1-771365e08b3a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:42.175778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-369a2e02-ed13-4f56-b1c1-771365e08b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:42.233580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a2e02-ed13-4f56-b1c1-771365e08b3a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:42.237338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd5c1-bbae-4820-b027-e4e7bb5c6388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:42.237975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd5c1-bbae-4820-b027-e4e7bb5c6388 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:42.238256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd5c1-bbae-4820-b027-e4e7bb5c6388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:42.238535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd5c1-bbae-4820-b027-e4e7bb5c6388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:42.247639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcd5c1-bbae-4820-b027-e4e7bb5c6388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:42.249091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcd5c1-bbae-4820-b027-e4e7bb5c6388 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:42.254079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:25:42.258832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 768/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:44.515642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad75759-40ed-4406-9c1b-7db25f57929c None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:44.516249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad75759-40ed-4406-9c1b-7db25f57929c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:44.516249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad75759-40ed-4406-9c1b-7db25f57929c None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:44.517593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:44.522975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f4dd8-7fe0-4310-a86f-a9e99c66336d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:44.524049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f4dd8-7fe0-4310-a86f-a9e99c66336d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:44.524281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f4dd8-7fe0-4310-a86f-a9e99c66336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:44.542798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-727f4dd8-7fe0-4310-a86f-a9e99c66336d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:44.544014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-727f4dd8-7fe0-4310-a86f-a9e99c66336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:44.551546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:44.587771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad75759-40ed-4406-9c1b-7db25f57929c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:44.589617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0ad75759-40ed-4406-9c1b-7db25f57929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:44.604152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad75759-40ed-4406-9c1b-7db25f57929c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:44.607678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374730c-b6c6-47c7-9d0a-17373c8f2c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:44.608317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374730c-b6c6-47c7-9d0a-17373c8f2c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:44.608601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374730c-b6c6-47c7-9d0a-17373c8f2c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:44.608879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374730c-b6c6-47c7-9d0a-17373c8f2c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:44.618267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374730c-b6c6-47c7-9d0a-17373c8f2c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:44.619757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374730c-b6c6-47c7-9d0a-17373c8f2c95 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:44.629744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 764/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:48.295085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bbac5-70e6-45de-8516-434e24317fe5 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:48.295698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bbac5-70e6-45de-8516-434e24317fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:48.295698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bbac5-70e6-45de-8516-434e24317fe5 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:48.297091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:48.302422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b3402-2b83-4164-8e9d-694aba62dc1b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:48.302843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b3402-2b83-4164-8e9d-694aba62dc1b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:48.302989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b3402-2b83-4164-8e9d-694aba62dc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:48.309365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=62 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:25:48.321188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-852b3402-2b83-4164-8e9d-694aba62dc1b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:48.327356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-852b3402-2b83-4164-8e9d-694aba62dc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:48.330447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:48.606023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bbac5-70e6-45de-8516-434e24317fe5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:48.607879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f16bbac5-70e6-45de-8516-434e24317fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:48.669639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bbac5-70e6-45de-8516-434e24317fe5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:48.673142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185187af-5142-4660-ad23-3cf5a7e4d884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:48.673789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185187af-5142-4660-ad23-3cf5a7e4d884 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:48.674085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185187af-5142-4660-ad23-3cf5a7e4d884 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:48.674389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185187af-5142-4660-ad23-3cf5a7e4d884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:48.683594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185187af-5142-4660-ad23-3cf5a7e4d884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:48.685025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185187af-5142-4660-ad23-3cf5a7e4d884 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:48.694464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 771/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:25:51.016597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde82f-463e-4525-a262-d094640b3924 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:51.017117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde82f-463e-4525-a262-d094640b3924 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:51.017117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde82f-463e-4525-a262-d094640b3924 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:51.018447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 1 04:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:25:51.024412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f788a-c902-4485-bd76-f9d80c502ea3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:51.024819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f788a-c902-4485-bd76-f9d80c502ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:51.024978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f788a-c902-4485-bd76-f9d80c502ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:51.043736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ba1f788a-c902-4485-bd76-f9d80c502ea3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:25:51.044996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ba1f788a-c902-4485-bd76-f9d80c502ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:51.052857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 1 04:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:25:51.326913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cde82f-463e-4525-a262-d094640b3924 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:25:51.328989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-90cde82f-463e-4525-a262-d094640b3924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:25:51.343568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cde82f-463e-4525-a262-d094640b3924 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:25:51.347098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958a4e3-9484-4693-82d5-fbb84fdfe52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:25:51.347784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a4e3-9484-4693-82d5-fbb84fdfe52d service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:25:51.348073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a4e3-9484-4693-82d5-fbb84fdfe52d service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:25:51.348352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a4e3-9484-4693-82d5-fbb84fdfe52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:25:51.358147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958a4e3-9484-4693-82d5-fbb84fdfe52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:25:51.359619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958a4e3-9484-4693-82d5-fbb84fdfe52d service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:25:51.369699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 767/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:26:01.038036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=32 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:26:01.365353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=8 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:23.251779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.252399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.252399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.301450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.302686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.316032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 773/1540] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:23.325001 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.325592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.325592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.356059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.358055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.369646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 768/1541] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.374990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.389059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a751f-82f5-4c7e-85cf-8e27c5c519d6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.400113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.400815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.401279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.401558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.402350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:23.402660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.404952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987b8a3-d92e-41f8-8b18-be5a1f3e42ae admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.427424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 774/1542] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:23.436389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.450570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeb560-1c37-4d0d-9015-329617f2a701 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.458740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4f76e-ba78-4d0c-b187-17ef3dd7a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.459994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4f76e-ba78-4d0c-b187-17ef3dd7a903 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.460489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4f76e-ba78-4d0c-b187-17ef3dd7a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.460979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4f76e-ba78-4d0c-b187-17ef3dd7a903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.461919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4f76e-ba78-4d0c-b187-17ef3dd7a903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.464083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4f76e-ba78-4d0c-b187-17ef3dd7a903 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.512834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485da7-433a-40bc-adce-acd23789401e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.513130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485da7-433a-40bc-adce-acd23789401e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.513606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89485da7-433a-40bc-adce-acd23789401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.541282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-89485da7-433a-40bc-adce-acd23789401e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.543322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-89485da7-433a-40bc-adce-acd23789401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.557517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 769/1543] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:23.559825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 775/1544] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.563929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485da7-433a-40bc-adce-acd23789401e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.570493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a1c36-e716-48b2-9347-728b23993708 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.571062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a1c36-e716-48b2-9347-728b23993708 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.571569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a1c36-e716-48b2-9347-728b23993708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.577447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89485da7-433a-40bc-adce-acd23789401e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.596238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-978a1c36-e716-48b2-9347-728b23993708 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.597765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-978a1c36-e716-48b2-9347-728b23993708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.689383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ac1d9-0150-4dba-a440-fcec37852ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.690185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ac1d9-0150-4dba-a440-fcec37852ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.690481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ac1d9-0150-4dba-a440-fcec37852ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.690764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ac1d9-0150-4dba-a440-fcec37852ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.691516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ac1d9-0150-4dba-a440-fcec37852ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.693596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ac1d9-0150-4dba-a440-fcec37852ff8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.694312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 770/1545] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:23.696194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c835-1a45-4b7a-a4f5-b4704585a570 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.696736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c835-1a45-4b7a-a4f5-b4704585a570 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.697234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c835-1a45-4b7a-a4f5-b4704585a570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.717415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ca45c835-1a45-4b7a-a4f5-b4704585a570 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.719519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ca45c835-1a45-4b7a-a4f5-b4704585a570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.728071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 771/1546] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.728246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5b70c-33d4-4bb0-a1fd-f16c2a98369c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.728455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5b70c-33d4-4bb0-a1fd-f16c2a98369c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.728740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5b70c-33d4-4bb0-a1fd-f16c2a98369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.759455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3bb5b70c-33d4-4bb0-a1fd-f16c2a98369c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.760659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3bb5b70c-33d4-4bb0-a1fd-f16c2a98369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.770732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 772/1547] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.771874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.772164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.772445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.775473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 776/1548] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:23.780501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980537-0717-4745-99fe-641757b143fd None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.780859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980537-0717-4745-99fe-641757b143fd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.781154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1980537-0717-4745-99fe-641757b143fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.793708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.798821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.802700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f1980537-0717-4745-99fe-641757b143fd None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.803958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f1980537-0717-4745-99fe-641757b143fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.804445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 773/1549] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.805347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.813522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 777/1550] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.814947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799878f-5bdf-4b95-9841-34e814511d41 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.815244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799878f-5bdf-4b95-9841-34e814511d41 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.815560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799878f-5bdf-4b95-9841-34e814511d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.815925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154ae14-9f58-4c02-8c3b-bdf761527917 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.820553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.821211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.821493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.821773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.822827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:23.823147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.824750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:27:23.824750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:27:23.824750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:27:23.827785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b3ec0-7a65-4c53-baf6-42324fd25814 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.834856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-4799878f-5bdf-4b95-9841-34e814511d41 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.836086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4799878f-5bdf-4b95-9841-34e814511d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.837365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 774/1551] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:23.838709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.838994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.839266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.847373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 778/1552] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.851768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050468b4-bdc1-456a-97e6-af79c6ea09a8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.852069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050468b4-bdc1-456a-97e6-af79c6ea09a8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.852358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050468b4-bdc1-456a-97e6-af79c6ea09a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.858958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.860260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.868724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 775/1553] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.869526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.877063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-050468b4-bdc1-456a-97e6-af79c6ea09a8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.878282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-050468b4-bdc1-456a-97e6-af79c6ea09a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.879776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5806a-aa10-4b10-900c-3f9e5478fd38 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.884299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87b88c-952c-42ec-ab59-d1508c138371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.884955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87b88c-952c-42ec-ab59-d1508c138371 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.885244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87b88c-952c-42ec-ab59-d1508c138371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.885526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87b88c-952c-42ec-ab59-d1508c138371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.886307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87b88c-952c-42ec-ab59-d1508c138371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.887157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 779/1554] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.888183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed16bc-32c7-422d-a7bb-c6a72af05924 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.888452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed16bc-32c7-422d-a7bb-c6a72af05924 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.888555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87b88c-952c-42ec-ab59-d1508c138371 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.888733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed16bc-32c7-422d-a7bb-c6a72af05924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.898950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 776/1555] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:23.900162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.900455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.900737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.907540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-53ed16bc-32c7-422d-a7bb-c6a72af05924 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.908733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-53ed16bc-32c7-422d-a7bb-c6a72af05924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.916566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 780/1556] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.918500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.918791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.919072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.921767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.922996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.932289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 777/1557] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.933742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.938363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:23.939609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:23.943203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db91760-8917-4e4d-9120-c9d96c8693c3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.948053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 781/1558] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:23.954806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a72000-a193-449e-914a-1aff483e36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.954992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:23.955144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72000-a193-449e-914a-1aff483e36e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.955144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72000-a193-449e-914a-1aff483e36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.955144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a72000-a193-449e-914a-1aff483e36e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.955144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72000-a193-449e-914a-1aff483e36e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.955144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a72000-a193-449e-914a-1aff483e36e6 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.960429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df879e1f-1b9b-45eb-979f-b5802c4f5bc8 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:23.965450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:23.966107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.966392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.966674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:23.967433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:23.967771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:23.969357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43859c-b51f-4afb-a39e-1450f427a4a7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:23.981699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 782/1559] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:23.983220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:23.983557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:23.983820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.002018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:24.003216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:24.011094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 783/1560] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:24.011922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.021159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ddd9b-8466-4b6a-9a87-9346d1def7bd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.025652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.026275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.026561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.026836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.027573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:24.027880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.029467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.040618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 784/1561] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:24.041390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.051351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155a18f-e9be-4eed-a8e3-8d643e094ab9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.055983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.056636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.056922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.057205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.057908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.060276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.072099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 778/1562] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:23 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.072099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.072099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.072099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.072551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 785/1563] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:24.072551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.083315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadbe94-44f3-477d-b713-aa2eb5cb70aa admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.090416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:24.091977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:24.129781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:24.142080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.142771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.143058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.143358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.143477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:24.144080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:24.144383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.145990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.155716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 779/1564] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:24.164373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.164484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 786/1565] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:24.164484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.165750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6698-97ff-42c3-8d1f-e3a18370c967 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.171159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.171852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.172144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.172426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.173207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:24.173516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.173912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0fd0c-3c96-4e2e-bc76-e04c010433e8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.175127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.178402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.179037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.179345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.179631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.180331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.181705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:27:24.181705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:27:24.181705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:27:24.183836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.187657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 780/1566] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:24.189713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.199521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e0a03-0d1d-450e-a90b-d8f2d3c904cc admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.204178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.204889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.205178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.205463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.206190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:24.206542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.208231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.220690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 781/1567] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:24.221941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.232736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65f57d-8266-4283-afeb-cc341b8ffd9f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.237296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.237920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.238204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.238482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.239157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.241061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.331630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 787/1568] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/projects => generated 355 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.333084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.354480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 782/1569] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.355864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.466336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9d9b9-1b2e-466a-b059-e6c397067fd5 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.466599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a9a9a-9a45-492d-80fc-f07f20827e13 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.471036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.471787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.471974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.472246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.472457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.472555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] PATH_INFO: `/v3/projects/d0ba5acb4aa042c39432f545d86619c6/users/366cb9679702412fa3bcf582cb1d4c1e/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.472746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.473027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.473716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.475730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.488345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ba5acb4aa042c39432f545d86619c6, user_id=366cb9679702412fa3bcf582cb1d4c1e, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.491052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.577367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 788/1570] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.578368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.587987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f4367-a01a-4fd8-9204-cb332892489e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.593108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.593728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.594016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.594294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.595000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:24.595324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.596927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.611088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 783/1571] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:24 2022] PUT /identity/v3/projects/d0ba5acb4aa042c39432f545d86619c6/users/366cb9679702412fa3bcf582cb1d4c1e/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:24.611088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.611370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 789/1572] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:24.611801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.637265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9b1f9-ccc6-4638-94dc-2d4707170371 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.671364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e267b18-4cc3-43ce-9530-be9348978e76 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.676309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.676975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.677263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.677545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.678238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.680224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.695163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.695855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.696147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.696429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.697122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.699583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.767732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 784/1573] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/projects => generated 391 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.768630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.782092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:24.790845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 790/1574] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.793684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.813585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:24.833313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb68631-3f6e-4927-a0a9-ee71579a1bba admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.855509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c6bfb-234d-45c7-bb51-e34ead30f087 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.861221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.861902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.862180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.862458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.863146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.865178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.891503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.892172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.892460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.892738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.893428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.895352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.958337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 791/1575] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/users => generated 278 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.959282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:24.969220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b717761-fff1-41b8-8d4c-922af7fea2aa admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:24.973629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613a952-ede6-4d69-ac8b-838fbed0d0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:24.974255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613a952-ede6-4d69-ac8b-838fbed0d0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:24.974537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613a952-ede6-4d69-ac8b-838fbed0d0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:24.974874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613a952-ede6-4d69-ac8b-838fbed0d0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:24.975507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613a952-ede6-4d69-ac8b-838fbed0d0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:24.977415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613a952-ede6-4d69-ac8b-838fbed0d0a5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:24.989230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 785/1576] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/projects => generated 405 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:24.990245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.051349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc5d9-9fe3-467b-a112-1f3772d989e2 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.056089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.056745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.057035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.057309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.058030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.060267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.070146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 786/1577] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.070847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.076442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 792/1578] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:24 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.076442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.076442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.076442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.081709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32bf1b-b414-4357-9659-791f09211c9a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.086530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.087177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.087495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.087771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.088457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.090977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.102046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] MFA Rules not processed for user `366cb9679702412fa3bcf582cb1d4c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:25.103250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:25.170457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 793/1579] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:25.171627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.171739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 787/1580] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.172441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.184421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f28415-dad6-4923-9fd8-c09b3decb184 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.193494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:25.235851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19c4a0-397b-4f2e-80c3-68b1e5ee93c4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.240655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.241330 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.241619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.241900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.242592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.243129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.243654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:27:25.243654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:27:25.243654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:27:25.243919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.244101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.244454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.244859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.245125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.247096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.337267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 788/1581] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.338274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.348239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9434-9e55-41ca-82f5-fb39bba90fed admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.352707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.353345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.353625 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.353906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.354618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.356825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.366788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 794/1582] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/users => generated 296 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.367270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 789/1583] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.368110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.368512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.378561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d636e83-6bdf-4175-aed8-d48af583a8ec admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.379366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598dc06-5665-4044-9ab8-ddc7c505408c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.383120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.383854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.383985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.384138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.384416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.384572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.384860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.385135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.385256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.385848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.386723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.387964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.392387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:25.397938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 795/1584] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.401396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.408930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fffb5-d98e-41e2-9d21-2ac2914ce601 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.413638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.414268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.414549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.414825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.415571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.417144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.428956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 796/1585] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.429775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.439206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6e106-c27e-4ebe-b663-8b53a5110968 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.443819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.444478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.444766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.445046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.445711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.447592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.495775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 790/1586] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/users => generated 303 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.496725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.506019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19acead-b689-4aed-92b6-86c38cc7b598 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.510526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.511160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.511960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.512236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.512908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.514807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.567814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 797/1587] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.567814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.598644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 791/1588] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.598644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.606331 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7a2d-d8e3-42d9-85d6-7611f49c33a8 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.631708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133c5ca-697e-49d5-82a2-eb45c0f0358d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.637014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.637780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.637999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.638239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.638969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.640611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.650629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 798/1589] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.651459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.661701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdfba5-19bf-464b-8cc4-a7bf54c69131 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.666385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.667028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.667340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.667646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.668362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.669954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.672137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.672611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.672964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.673306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.674107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.675788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.679984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 799/1590] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.680771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.689326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 792/1591] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.689326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.690950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44177faf-1918-423c-9812-24c1b56c13f9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.695689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.696548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.696836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.696945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5867da-6afb-405d-aded-a70543aabeaa admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.697117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] PATH_INFO: `/v3/projects/7cef434ad11449bf91a4e809384fd02a/users/1564a394b8b74a9fab20facd7f91a125/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.702090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.702700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.702984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.703260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.704005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.705734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.715608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cef434ad11449bf91a4e809384fd02a, user_id=1564a394b8b74a9fab20facd7f91a125, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.716665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:27:25.716665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:27:25.716665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:27:25.720267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 793/1592] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.720267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.720483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.727574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a588f2-04db-44c5-a640-10d266d5a3de admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.732207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.733104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.733387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.733662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.734376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.735984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.746754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 794/1593] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.747617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.756633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f483c65-f771-49ce-9528-8be227cc52a1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.761684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.762306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.762587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.762859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.763616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.765502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.794479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 800/1594] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:25 2022] PUT /identity/v3/projects/7cef434ad11449bf91a4e809384fd02a/users/1564a394b8b74a9fab20facd7f91a125/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:25.795325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.856295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea0052-6e28-42ae-8934-557049fb86f4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.861421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.862113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.862398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.862676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.863434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.865044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.867043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 795/1595] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:25 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:25.867723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.877622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bae903-79de-426c-bc88-fd729e569507 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.881294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 801/1596] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.881294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.882326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.882987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.883679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.883951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.884674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.886265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.889625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f94b2c-98d3-4753-b58a-ed08d49d5593 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.894257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.895119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.895439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.895869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] PATH_INFO: `/v3/projects/ff19af64e879451ea621d313631390c4/users/2af32f291664436ea90593be86310003/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.897561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 796/1597] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.899554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.908794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc37df-63f6-47a7-bb9b-3a5a71f303b7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.910762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff19af64e879451ea621d313631390c4, user_id=2af32f291664436ea90593be86310003, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.913402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.913558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.914059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.914346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.914627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.915381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.916965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.919088 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:25.927613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 797/1598] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:25.928993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:25.938544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6066a-5071-46a6-8746-d9544478b0c4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:25.943069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9750c-1658-4d1d-9e54-9f0f0f595bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:25.943965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9750c-1658-4d1d-9e54-9f0f0f595bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:25.944251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9750c-1658-4d1d-9e54-9f0f0f595bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:25.944561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9750c-1658-4d1d-9e54-9f0f0f595bd5 admin admin] PATH_INFO: `/v3/projects/7a45f5b11ffc4a99b4d7d7b8fbf86838/users/4801021c8f4741d9952c5db2f0e22f1a/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:25.959256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9750c-1658-4d1d-9e54-9f0f0f595bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a45f5b11ffc4a99b4d7d7b8fbf86838, user_id=4801021c8f4741d9952c5db2f0e22f1a, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:25.962017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9750c-1658-4d1d-9e54-9f0f0f595bd5 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:25.982666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:25.987559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 802/1599] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:25 2022] PUT /identity/v3/projects/ff19af64e879451ea621d313631390c4/users/2af32f291664436ea90593be86310003/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:25.989404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.049622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25d1b-e774-4059-9e23-25930a180a57 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.055573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 798/1600] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:25 2022] PUT /identity/v3/projects/7a45f5b11ffc4a99b4d7d7b8fbf86838/users/4801021c8f4741d9952c5db2f0e22f1a/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:26.067370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.067370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.067370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.088406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] MFA Rules not processed for user `1564a394b8b74a9fab20facd7f91a125`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.089616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.117799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.118507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.118793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.119072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.119826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.121964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.131677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 803/1601] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:26.132499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.145018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cce0b-7714-4fd1-b2da-1a1599454cc2 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.149669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.150322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.150611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.150891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.151667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.153299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.155021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 799/1602] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.155909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.163982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 804/1603] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:26.164807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.166068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad0568-20b9-4456-a017-d8d0acf1eca5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.171268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.172120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.172264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.172531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.173264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.174881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.176252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c47909-8322-4a65-adec-b532fd52cf65 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.181255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b6e68-8b33-4277-96d9-0fc7b0fdfd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.182132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b6e68-8b33-4277-96d9-0fc7b0fdfd61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.182455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b6e68-8b33-4277-96d9-0fc7b0fdfd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.182809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b6e68-8b33-4277-96d9-0fc7b0fdfd61 admin admin] PATH_INFO: `/v3/projects/d12a9b5dc38b4aa7aed77adc89d0e6a4/users/85df2f6595d3489bb5d1d547cb2cd900/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.190841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 800/1604] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:26.191500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.201295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b6e68-8b33-4277-96d9-0fc7b0fdfd61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12a9b5dc38b4aa7aed77adc89d0e6a4, user_id=85df2f6595d3489bb5d1d547cb2cd900, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.203611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dee73f-12db-4a7d-979d-00d3e5b5585c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.204478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b6e68-8b33-4277-96d9-0fc7b0fdfd61 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.206665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b7134-58d3-4012-9061-96f838b43e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.207693 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b7134-58d3-4012-9061-96f838b43e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.207992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b7134-58d3-4012-9061-96f838b43e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.208269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b7134-58d3-4012-9061-96f838b43e44 admin admin] PATH_INFO: `/v3/projects/bda53dca70e54e208b2767a833923234/users/12816eca659542af9e41b92fdb674eab/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.223290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b7134-58d3-4012-9061-96f838b43e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda53dca70e54e208b2767a833923234, user_id=12816eca659542af9e41b92fdb674eab, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.226751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b7134-58d3-4012-9061-96f838b43e44 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.309649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 805/1605] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:26 2022] PUT /identity/v3/projects/d12a9b5dc38b4aa7aed77adc89d0e6a4/users/85df2f6595d3489bb5d1d547cb2cd900/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:26.311237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.311237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.311497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.319920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 801/1606] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:26 2022] PUT /identity/v3/projects/bda53dca70e54e208b2767a833923234/users/12816eca659542af9e41b92fdb674eab/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:26.321371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.321732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.322082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.337996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] MFA Rules not processed for user `2af32f291664436ea90593be86310003`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.339195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.349030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] MFA Rules not processed for user `4801021c8f4741d9952c5db2f0e22f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.350212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.400029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:26.405367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 806/1607] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.406522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.419607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 802/1608] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.420826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.442002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:26.478114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d26505-888d-43e3-8392-846dd5abde82 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.483989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd57c36-7234-46bf-89b8-e6d90870b574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.484879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd57c36-7234-46bf-89b8-e6d90870b574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.485164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd57c36-7234-46bf-89b8-e6d90870b574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.485439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd57c36-7234-46bf-89b8-e6d90870b574 admin admin] PATH_INFO: `/v3/projects/55c8b31a3d3e45e1b87985d948cec440/users/72d74d4eb3384eec8511b485cf99db09/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.487526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d795aa-500e-4d0d-aac2-63316c3accee None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.492285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029092f2-de88-46ee-addb-6b67348b466a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.493177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029092f2-de88-46ee-addb-6b67348b466a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.493462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029092f2-de88-46ee-addb-6b67348b466a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.493737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029092f2-de88-46ee-addb-6b67348b466a admin admin] PATH_INFO: `/v3/projects/eb47091723aa4c1db05243cdf458e529/users/6785df5e54c548c6a6a42137c6c412ae/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.501891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd57c36-7234-46bf-89b8-e6d90870b574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c8b31a3d3e45e1b87985d948cec440, user_id=72d74d4eb3384eec8511b485cf99db09, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.504598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd57c36-7234-46bf-89b8-e6d90870b574 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.508766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029092f2-de88-46ee-addb-6b67348b466a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb47091723aa4c1db05243cdf458e529, user_id=6785df5e54c548c6a6a42137c6c412ae, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.511452 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029092f2-de88-46ee-addb-6b67348b466a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.570046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 807/1609] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:26 2022] PUT /identity/v3/projects/55c8b31a3d3e45e1b87985d948cec440/users/72d74d4eb3384eec8511b485cf99db09/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:26.571608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fabe74-c089-473b-b57d-72daa751dec2 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.572483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fabe74-c089-473b-b57d-72daa751dec2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.572483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fabe74-c089-473b-b57d-72daa751dec2 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.573874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 808/1610] 10.222.0.18 () {54 vars in 961 bytes} [Wed Jun 1 04:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:26.574661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fabe74-c089-473b-b57d-72daa751dec2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.598371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 803/1611] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:26 2022] PUT /identity/v3/projects/eb47091723aa4c1db05243cdf458e529/users/6785df5e54c548c6a6a42137c6c412ae/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:26.599704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9aab4-c98f-40fb-bd19-7755a4f92c8f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.599996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9aab4-c98f-40fb-bd19-7755a4f92c8f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.600280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9aab4-c98f-40fb-bd19-7755a4f92c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.623329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-70d9aab4-c98f-40fb-bd19-7755a4f92c8f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.625195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-70d9aab4-c98f-40fb-bd19-7755a4f92c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.650688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fabe74-c089-473b-b57d-72daa751dec2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.687215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 804/1612] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.688714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.689105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.689498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.711159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.711159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.711448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418c760-7142-4e3e-9d1c-ddc462f55345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.711570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418c760-7142-4e3e-9d1c-ddc462f55345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.711814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418c760-7142-4e3e-9d1c-ddc462f55345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.712100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418c760-7142-4e3e-9d1c-ddc462f55345 admin admin] PATH_INFO: `/v3/projects/8eb15379902c4745a1e3fd5b83d0e4be/users/c2eb70d1582b4876931f05cd4015a346/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.718152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 805/1613] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.719060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.728625 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8495-7a52-4370-b0f3-6253fba1ce9b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:26.731919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418c760-7142-4e3e-9d1c-ddc462f55345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb15379902c4745a1e3fd5b83d0e4be, user_id=c2eb70d1582b4876931f05cd4015a346, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.734903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418c760-7142-4e3e-9d1c-ddc462f55345 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.735347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a52e5f-4bba-47b8-a311-39b34783ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:26.736009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a52e5f-4bba-47b8-a311-39b34783ddeb admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.736294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a52e5f-4bba-47b8-a311-39b34783ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.736570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a52e5f-4bba-47b8-a311-39b34783ddeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.737255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a52e5f-4bba-47b8-a311-39b34783ddeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:26.739194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a52e5f-4bba-47b8-a311-39b34783ddeb admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:26.832747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 809/1614] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:26 2022] PUT /identity/v3/projects/8eb15379902c4745a1e3fd5b83d0e4be/users/c2eb70d1582b4876931f05cd4015a346/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:26.834076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca7d9-38b0-40d7-b5bb-526923f112ba None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.834456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca7d9-38b0-40d7-b5bb-526923f112ba None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.834740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca7d9-38b0-40d7-b5bb-526923f112ba None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.836090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 810/1615] 10.222.0.18 () {54 vars in 961 bytes} [Wed Jun 1 04:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:26.837364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.837649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.837923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.846770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 806/1616] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/projects => generated 357 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:26.848148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:26.848553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:26.848857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:26.858701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:26.866670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] MFA Rules not processed for user `72d74d4eb3384eec8511b485cf99db09`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.868274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.868416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:26.869349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:26.937638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 811/1617] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.938489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.949335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 807/1618] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Jun 1 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:26.950689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:26.956607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:27.008059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319e41f-d8de-44ca-8e7c-314642fe4852 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.010679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1e22c-906c-4226-a48d-84018a13b12f None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.011562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34ec65-f412-4d9e-ad62-cfe07aa99fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.012743 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34ec65-f412-4d9e-ad62-cfe07aa99fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.013024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34ec65-f412-4d9e-ad62-cfe07aa99fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.013301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34ec65-f412-4d9e-ad62-cfe07aa99fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.016210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.019529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.019529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.019529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.019529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.020025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.022507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34ec65-f412-4d9e-ad62-cfe07aa99fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.023954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34ec65-f412-4d9e-ad62-cfe07aa99fe6 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.030518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 812/1619] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:27.031353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.034497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 808/1620] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:27.035938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63990f71-b0f3-4b51-8ba1-3852fb84ace0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.036304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63990f71-b0f3-4b51-8ba1-3852fb84ace0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.036726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63990f71-b0f3-4b51-8ba1-3852fb84ace0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.041824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa40ba1-9c8d-4888-a768-ed12a48d4a1d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.046454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.047211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.047621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.047992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.055363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-63990f71-b0f3-4b51-8ba1-3852fb84ace0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:27.056638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-63990f71-b0f3-4b51-8ba1-3852fb84ace0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:27.058544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.060211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.065140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 809/1621] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:27.066456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.066750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.067114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.070628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 813/1622] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:27.071539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.081717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e79dd-c7ba-446c-80dd-33da14360a9e service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.087024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:27.088256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:27.088493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.089148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.089436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.090656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.091017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.092992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.098900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 810/1623] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Jun 1 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:27.099706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.111346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68e31-705a-433c-bfa8-6b005e7a06f4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.115824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.116477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.116760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.117038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.117736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.119679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.189310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 814/1624] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:27 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:27.190282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.203271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce448eb-a1fc-4221-ab1f-4faec7e89079 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.208942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.209614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.209915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.210198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.239664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 811/1625] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:27 2022] POST /identity/v3/projects => generated 399 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:27.239664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.286248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.288636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.299774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 815/1626] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:27.300588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.306499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:27.311150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ebaf4-cc95-4741-ab1f-62b7a1281434 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.316620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b577e59-a722-4a6b-ae24-0ceee5d4a041 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.322144 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.323335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.323700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.324061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] PATH_INFO: `/v3/projects/eb47091723aa4c1db05243cdf458e529/users/6785df5e54c548c6a6a42137c6c412ae/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.326927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb47091723aa4c1db05243cdf458e529, user_id=6785df5e54c548c6a6a42137c6c412ae, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.329684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.350783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 812/1627] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:27 2022] PUT /identity/v3/projects/eb47091723aa4c1db05243cdf458e529/users/6785df5e54c548c6a6a42137c6c412ae/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:27.352926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.393091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.393983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.400003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.400003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.406033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.408008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.422249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 816/1628] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:27.423623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.429685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:27.435870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450cad5-2444-4bca-a3a6-6a1e262437f6 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.436486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e6e7c-0fac-4d2c-88f3-2f4e201d8663 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.446393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c67c1-edd3-4d1c-8bf1-8b5ac145efdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.446768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c67c1-edd3-4d1c-8bf1-8b5ac145efdd service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.447146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c67c1-edd3-4d1c-8bf1-8b5ac145efdd service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.447547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c67c1-edd3-4d1c-8bf1-8b5ac145efdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.515418 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.516431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.516642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.516932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.518069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.519792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.538482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 817/1629] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:27.539617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.551503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c67c1-edd3-4d1c-8bf1-8b5ac145efdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.551503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c67c1-edd3-4d1c-8bf1-8b5ac145efdd service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.552958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c624cec-017f-4cce-abe4-0914a553069b admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.560769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.561460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.561933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.562222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.562805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 813/1630] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:27.564523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.564921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.565301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.576685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.582966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.598085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] MFA Rules not processed for user `6785df5e54c548c6a6a42137c6c412ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:27.600084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:27.696281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 818/1631] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:27 2022] POST /identity/v3/users => generated 306 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:27.697308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.725169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9fb22-3cf5-406f-a645-a49b0f195854 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.740076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.741242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.741769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.742255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] PATH_INFO: `/v3/projects/d57f667939ed4342a27068dee822ed7f/users/c39670d3940b4a21beb9f22bd5936dc7/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.761465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57f667939ed4342a27068dee822ed7f, user_id=c39670d3940b4a21beb9f22bd5936dc7, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.764251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.770742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 814/1632] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:27.770742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.782594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bd107-f91f-4893-b2ae-6f298ac4c9ef None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.787768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365797-0bfa-413f-a6f4-8b07b060788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.788733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365797-0bfa-413f-a6f4-8b07b060788d admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.789137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365797-0bfa-413f-a6f4-8b07b060788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.789519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365797-0bfa-413f-a6f4-8b07b060788d admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.790560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365797-0bfa-413f-a6f4-8b07b060788d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.792167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365797-0bfa-413f-a6f4-8b07b060788d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.807326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 815/1633] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:27.807326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8930d5c-f19e-429b-8f84-7edc90c474e0 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.807326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8930d5c-f19e-429b-8f84-7edc90c474e0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.807326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8930d5c-f19e-429b-8f84-7edc90c474e0 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.814265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 816/1634] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jun 1 04:27:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:27.815381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8930d5c-f19e-429b-8f84-7edc90c474e0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.845047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 819/1635] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:27 2022] PUT /identity/v3/projects/d57f667939ed4342a27068dee822ed7f/users/c39670d3940b4a21beb9f22bd5936dc7/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:27.846335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:27.914002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8930d5c-f19e-429b-8f84-7edc90c474e0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.933217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7192c-6cd3-49f5-a185-b21f06424022 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:27.939637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c88d8-0d4c-484e-b9b0-eb39b7d10587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:27.940830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c88d8-0d4c-484e-b9b0-eb39b7d10587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:27.941177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c88d8-0d4c-484e-b9b0-eb39b7d10587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:27.941580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c88d8-0d4c-484e-b9b0-eb39b7d10587 admin admin] PATH_INFO: `/v3/projects/8e17a41e261f40088f9ae753780e6f01/users/5c8dac68c90544159d4670fa061006a9/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:27.945257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:27.969057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c88d8-0d4c-484e-b9b0-eb39b7d10587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e17a41e261f40088f9ae753780e6f01, user_id=5c8dac68c90544159d4670fa061006a9, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:27.983735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c88d8-0d4c-484e-b9b0-eb39b7d10587 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:27.985848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:28.007908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.008947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.009550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.010630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.077663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 820/1636] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:27 2022] PUT /identity/v3/projects/8e17a41e261f40088f9ae753780e6f01/users/5c8dac68c90544159d4670fa061006a9/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:28.077663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d336ab-971e-4529-8465-0eede0a52d35 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.077663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d336ab-971e-4529-8465-0eede0a52d35 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.077663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d336ab-971e-4529-8465-0eede0a52d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.100186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.101913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.114763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 817/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:28.115758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.115929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d3d336ab-971e-4529-8465-0eede0a52d35 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:28.117743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d3d336ab-971e-4529-8465-0eede0a52d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:28.133205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e7dc0-69ea-4a88-b6b6-66213bd8130b service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.227916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 821/1638] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:28.229854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906834b0-4c73-4979-865b-9bacb42602ef None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.230277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906834b0-4c73-4979-865b-9bacb42602ef None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.235725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906834b0-4c73-4979-865b-9bacb42602ef None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.235725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 822/1639] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jun 1 04:27:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:28.235725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906834b0-4c73-4979-865b-9bacb42602ef None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.236024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05418-dc03-4de9-a116-e4ee678a67f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.236024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05418-dc03-4de9-a116-e4ee678a67f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.236024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05418-dc03-4de9-a116-e4ee678a67f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.236372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05418-dc03-4de9-a116-e4ee678a67f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.237118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05418-dc03-4de9-a116-e4ee678a67f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.243659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05418-dc03-4de9-a116-e4ee678a67f0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.252940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906834b0-4c73-4979-865b-9bacb42602ef None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.258997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 818/1640] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:28.260648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.264673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.264673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.265118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.265118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.265118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.265118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.265628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.268940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.282951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 823/1641] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:28.284026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.285118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:28.286361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:28.297251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 819/1642] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:28.298102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.311335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01636f9b-1724-4c7e-a041-1352eaa5b7b8 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.316310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.317399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.322047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.322047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] PATH_INFO: `/v3/projects/d57f667939ed4342a27068dee822ed7f/users/c39670d3940b4a21beb9f22bd5936dc7/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.322047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57f667939ed4342a27068dee822ed7f, user_id=c39670d3940b4a21beb9f22bd5936dc7, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.323825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.336884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 820/1643] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:28 2022] PUT /identity/v3/projects/d57f667939ed4342a27068dee822ed7f/users/c39670d3940b4a21beb9f22bd5936dc7/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:28.341861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.353070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a99993-b2c7-4f99-9694-cbb7257b266e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.409582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.410537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.411023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.413763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.436622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb59dec-5cce-4c5c-9b3e-ecad531e1a66 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.441767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcf330-01f2-4b83-9b37-8dc120efca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.442864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcf330-01f2-4b83-9b37-8dc120efca15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.443030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcf330-01f2-4b83-9b37-8dc120efca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.443327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcf330-01f2-4b83-9b37-8dc120efca15 admin admin] PATH_INFO: `/v3/projects/8e17a41e261f40088f9ae753780e6f01/users/5c8dac68c90544159d4670fa061006a9/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.445462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcf330-01f2-4b83-9b37-8dc120efca15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e17a41e261f40088f9ae753780e6f01, user_id=5c8dac68c90544159d4670fa061006a9, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.448569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcf330-01f2-4b83-9b37-8dc120efca15 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.453481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:28.458192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:28.464481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 821/1644] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:28 2022] PUT /identity/v3/projects/8e17a41e261f40088f9ae753780e6f01/users/5c8dac68c90544159d4670fa061006a9/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:28.467122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25fe82-056f-4b92-a22d-80df79a03648 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.467722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25fe82-056f-4b92-a22d-80df79a03648 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.468154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25fe82-056f-4b92-a22d-80df79a03648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.523490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0e25fe82-056f-4b92-a22d-80df79a03648 None None] MFA Rules not processed for user `c39670d3940b4a21beb9f22bd5936dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:28.525109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0e25fe82-056f-4b92-a22d-80df79a03648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:28.556968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.559332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.579151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 824/1645] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:28.581837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.607782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec2522-e1f2-4b21-b4bd-78d098359c83 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.684163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 822/1646] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:28.691108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.691264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.696127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.735409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.736298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.736587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.736925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.741954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] MFA Rules not processed for user `5c8dac68c90544159d4670fa061006a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:28.748974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:28.826775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 823/1647] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:28.844125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:28.857047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ed424-842c-4185-84ff-4a21f0aec75a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:28.862628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:28.863386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:28.863678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:28.863963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:28.876834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.878311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.883962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:28.885705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:28.892783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 824/1648] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:28 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:27:28.910348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 825/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:29.087843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:29.104278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3088de-49ea-41ad-bb83-0093ac843752 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:29.114242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:29.115407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:29.115904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:29.116423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:29.127901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:29.129730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:29.142026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 825/1650] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:29.517371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:29.524180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb455-3325-4865-8e9e-84f2ae1d72a6 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:29.545796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:29.545796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:29.545796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:29.545796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] PATH_INFO: `/v3/users/1564a394b8b74a9fab20facd7f91a125` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:29.547931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1564a394b8b74a9fab20facd7f91a125)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:29.547931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: WARNING py.warnings [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:27:29.547931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: warnings.warn(msg) Jun 01 04:27:29.547931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: Jun 01 04:27:29.547931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:29.557353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:29.592240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:29.601544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d887de-1e69-4274-98ee-c13f9442db1a service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:29.611254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a056fa-8498-46fb-a110-c5d1573988e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:29.611946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a056fa-8498-46fb-a110-c5d1573988e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:29.612238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a056fa-8498-46fb-a110-c5d1573988e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:29.612522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a056fa-8498-46fb-a110-c5d1573988e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:29.688931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:29.719721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a056fa-8498-46fb-a110-c5d1573988e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:29.721703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a056fa-8498-46fb-a110-c5d1573988e2 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:29.733424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 826/1651] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:29 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:29.794764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1564a394b8b74a9fab20facd7f91a125'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:29.795525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1564a394b8b74a9fab20facd7f91a125'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:29.825033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1564a394b8b74a9fab20facd7f91a125'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:29.840293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1564a394b8b74a9fab20facd7f91a125'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:29.938061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-471dff39-ec63-454e-913e-4556cfb5b99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1564a394b8b74a9fab20facd7f91a125'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:29.949252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 826/1652] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:29 2022] DELETE /identity/v3/users/1564a394b8b74a9fab20facd7f91a125 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:29.954339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:29.954635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:29.954913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:29.989624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:29.991991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:30.073800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081f79-1a66-406b-9ea5-44ad765ee4e8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:30.074096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081f79-1a66-406b-9ea5-44ad765ee4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:30.074446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90081f79-1a66-406b-9ea5-44ad765ee4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:30.166087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-90081f79-1a66-406b-9ea5-44ad765ee4e8 None None] MFA Rules not processed for user `c2eb70d1582b4876931f05cd4015a346`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:30.169914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-90081f79-1a66-406b-9ea5-44ad765ee4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:30.173119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 827/1653] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:30.232332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:30.248080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:30.389524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 827/1654] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:30.404383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:30.404794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:30.405207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:30.409429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85f42c-a790-4463-9ffe-0f1e0bafa789 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:30.420312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:30.421040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:30.422073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:30.422362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:30.448522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:30.450747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:30.457846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] MFA Rules not processed for user `c2eb70d1582b4876931f05cd4015a346`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:30.459245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:30.477424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 828/1655] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:27:30.485612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 828/1656] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:30.499808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:30.507114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:30.527259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:30.530318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:30.562077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7f297-9c0d-4b79-bc0d-452f5cad959e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:30.569614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:30.575941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:30.575941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:30.575941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] PATH_INFO: `/v3/users/72d74d4eb3384eec8511b485cf99db09` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:30.575941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d74d4eb3384eec8511b485cf99db09)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:30.575941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:30.668756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0706d-e5b4-4fdb-9aed-3a213cdd9beb service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:30.674674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1d220-d142-46f0-98f0-c7c4f1f8ef0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:30.675394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1d220-d142-46f0-98f0-c7c4f1f8ef0c service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:30.675686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1d220-d142-46f0-98f0-c7c4f1f8ef0c service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:30.676005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1d220-d142-46f0-98f0-c7c4f1f8ef0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:30.687258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1d220-d142-46f0-98f0-c7c4f1f8ef0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:30.688927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1d220-d142-46f0-98f0-c7c4f1f8ef0c service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:30.701086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 829/1657] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:30.872607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d74d4eb3384eec8511b485cf99db09'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:30.873314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d74d4eb3384eec8511b485cf99db09'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:30.887332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:30.889586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d74d4eb3384eec8511b485cf99db09'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:30.898128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d74d4eb3384eec8511b485cf99db09'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:30.972756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d74d4eb3384eec8511b485cf99db09'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:30.989278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 829/1658] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:30 2022] DELETE /identity/v3/users/72d74d4eb3384eec8511b485cf99db09 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:30.995080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:30.995407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:30.995692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:31.006525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:31.032615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:31.034092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:31.216348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 830/1659] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:31.256893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:31.258657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:31.318402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:31.320964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:31.341327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ad6e9-53ec-4ae8-9aa5-bc20296f1b31 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:31.358408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:31.359391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:31.362777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:31.363093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] PATH_INFO: `/v3/users/366cb9679702412fa3bcf582cb1d4c1e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:31.372045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] RBAC: Authorizing `identity:delete_user(user_id=366cb9679702412fa3bcf582cb1d4c1e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:31.372045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:31.428710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ce7fa-da4a-4ae2-bd45-4a2c16bba40a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:31.445647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceed768-b77b-41cd-8acb-dfe9ca1b0ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:31.447912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed768-b77b-41cd-8acb-dfe9ca1b0ade service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:31.448666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed768-b77b-41cd-8acb-dfe9ca1b0ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:31.449472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceed768-b77b-41cd-8acb-dfe9ca1b0ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:31.478041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceed768-b77b-41cd-8acb-dfe9ca1b0ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:31.479797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceed768-b77b-41cd-8acb-dfe9ca1b0ade service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:31.504508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 830/1660] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:31.602069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3deefb-9325-41d3-9cea-99fe4910280d None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:31.602950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3deefb-9325-41d3-9cea-99fe4910280d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:31.603456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3deefb-9325-41d3-9cea-99fe4910280d None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:31.605346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 831/1661] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jun 1 04:27:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:31.614210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7fcff-84bd-4043-9d05-81219015c7a0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:31.614738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7fcff-84bd-4043-9d05-81219015c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:31.615188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7fcff-84bd-4043-9d05-81219015c7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:31.660324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366cb9679702412fa3bcf582cb1d4c1e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:31.701899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366cb9679702412fa3bcf582cb1d4c1e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:31.703382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366cb9679702412fa3bcf582cb1d4c1e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:31.719153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366cb9679702412fa3bcf582cb1d4c1e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:31.757770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c6b7fcff-84bd-4043-9d05-81219015c7a0 None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:31.762076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c6b7fcff-84bd-4043-9d05-81219015c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:31.784802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 832/1662] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:31.816399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366cb9679702412fa3bcf582cb1d4c1e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:31.825100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 831/1663] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:31 2022] DELETE /identity/v3/users/366cb9679702412fa3bcf582cb1d4c1e => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:31.831510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:31.831951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:31.832346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:31.857068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:31.858998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:31.873984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:31.878136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:31.994988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e36b4-1e55-45aa-821a-7991a6a812df admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:32.004032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:32.004745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.005783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.006232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.013301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 833/1664] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:32.014437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:32.015820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:32.016206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:32.019892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:32.031017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 832/1665] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:27:32.058070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd8e7e-a1c5-4d49-98c8-9dce4b331a15 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:32.066057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:32.068513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:32.079875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:32.080796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.081102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.081385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] PATH_INFO: `/v3/projects/7cef434ad11449bf91a4e809384fd02a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.087825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209ae22-21ae-4f60-953a-2227bdb3ab70 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:32.087942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cef434ad11449bf91a4e809384fd02a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:32.091167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:32.092430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eab665-1c3a-4690-83fa-87db4af494c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:32.093277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eab665-1c3a-4690-83fa-87db4af494c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.093706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eab665-1c3a-4690-83fa-87db4af494c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.094163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eab665-1c3a-4690-83fa-87db4af494c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.111932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eab665-1c3a-4690-83fa-87db4af494c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:32.113465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eab665-1c3a-4690-83fa-87db4af494c2 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:32.146806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 833/1666] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:32.279029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cef434ad11449bf91a4e809384fd02a', 'request_id': 'req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:32.287938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cef434ad11449bf91a4e809384fd02a', 'request_id': 'req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:32.309515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:32.395780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cef434ad11449bf91a4e809384fd02a', 'request_id': 'req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:32.414844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] The token cache is being invalidate because project 7cef434ad11449bf91a4e809384fd02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:32.415286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:32.416975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7e7665d2-222c-45d9-8a6a-8d4e8702a57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:32.418806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 834/1667] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:32 2022] DELETE /identity/v3/projects/7cef434ad11449bf91a4e809384fd02a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:32.424078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.424376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.424664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.452271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:32.462815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:32.464496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:32.558313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 834/1668] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:32.559938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.560233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.560521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.586575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:32.588188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:32.598269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 835/1669] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:32.603179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:32.622403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3e873-5ea3-40b3-b0de-f7ea381c540f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:32.635825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:32.636779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.638077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.638365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.639380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:32.639907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:32.641660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:32.672535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 835/1670] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:32.679003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:32.700511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea34db-46e1-4955-9d8f-320b6389d341 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:32.709087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:32.709805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:32.710254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:32.710683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:32.711588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:32.718842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:32.865789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 836/1671] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:32 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:32.869840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:32.881382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddfe63-4fba-47c4-b933-2972cda83031 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:33.024167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:33.025445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.025874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.026306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.027245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:33.033790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:33.153315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 836/1672] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:32 2022] POST /identity/v3/users => generated 291 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:33.153408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:33.164980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d24c2-cf9e-49fb-a4be-162e7864eaba admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:33.176262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:33.177503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.177978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.178444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.179602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:33.181746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:33.219684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:33.221475 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:33.233387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 837/1673] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:27:33.237334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:33.245015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252576b-329b-48fe-977d-cbb022f4fc27 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:33.257238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:33.258145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] PATH_INFO: `/v3/projects/d0ba5acb4aa042c39432f545d86619c6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ba5acb4aa042c39432f545d86619c6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: WARNING py.warnings [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: warnings.warn(msg) Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: Jun 01 04:27:33.266831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:33.276968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9690-48d7-468e-bf58-3c277afbe45a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:33.290203 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189d471-d3b0-4c95-bf7c-239c27554d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:33.290940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d471-d3b0-4c95-bf7c-239c27554d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.291227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d471-d3b0-4c95-bf7c-239c27554d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.291530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189d471-d3b0-4c95-bf7c-239c27554d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.300937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189d471-d3b0-4c95-bf7c-239c27554d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:33.300937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189d471-d3b0-4c95-bf7c-239c27554d5c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:33.325222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 838/1674] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:33.329625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bf356-788d-4527-a158-485f6aa936a3 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.329926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bf356-788d-4527-a158-485f6aa936a3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.330202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bf356-788d-4527-a158-485f6aa936a3 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.343500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 839/1675] 10.222.0.18 () {54 vars in 960 bytes} [Wed Jun 1 04:27:33 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:33.344562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bf356-788d-4527-a158-485f6aa936a3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:33.374309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bf356-788d-4527-a158-485f6aa936a3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:33.389874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad2a18-7c34-41cd-997c-2edade66dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:33.391588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad2a18-7c34-41cd-997c-2edade66dbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.392263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad2a18-7c34-41cd-997c-2edade66dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.392263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad2a18-7c34-41cd-997c-2edade66dbbd admin admin] PATH_INFO: `/v3/projects/9407fbe87482460a85db70f33ce89b31/users/cbd9c02d0eb3462da854b75745861101/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.425662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad2a18-7c34-41cd-997c-2edade66dbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9407fbe87482460a85db70f33ce89b31, user_id=cbd9c02d0eb3462da854b75745861101, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:33.432423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad2a18-7c34-41cd-997c-2edade66dbbd admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:33.486089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ba5acb4aa042c39432f545d86619c6', 'request_id': 'req-66948952-0821-45a5-80f6-a088c6eecefd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:33.522995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 840/1676] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:33 2022] PUT /identity/v3/projects/9407fbe87482460a85db70f33ce89b31/users/cbd9c02d0eb3462da854b75745861101/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:33.524659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.524960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.525294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.548632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:33.549917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:33.605876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ba5acb4aa042c39432f545d86619c6', 'request_id': 'req-66948952-0821-45a5-80f6-a088c6eecefd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:33.614250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ba5acb4aa042c39432f545d86619c6', 'request_id': 'req-66948952-0821-45a5-80f6-a088c6eecefd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:33.656495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] The token cache is being invalidate because project d0ba5acb4aa042c39432f545d86619c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:33.658896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:33.661610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66948952-0821-45a5-80f6-a088c6eecefd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:33.665900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 837/1677] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:33 2022] DELETE /identity/v3/projects/d0ba5acb4aa042c39432f545d86619c6 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:33.670278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.670848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.671139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.693547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 841/1678] 10.222.0.18 () {58 vars in 1057 bytes} [Wed Jun 1 04:27:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:33.702392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:33.703928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:33.729334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] MFA Rules not processed for user `cbd9c02d0eb3462da854b75745861101`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:33.733463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:33.862687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a106-13de-4d26-9b57-9f3f6dfe00b1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:33.878206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:33.879410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:33.879867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:33.880834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] PATH_INFO: `/v3/projects/55c8b31a3d3e45e1b87985d948cec440` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:33.882251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c8b31a3d3e45e1b87985d948cec440)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:33.888214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:33.943676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 838/1679] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:33.945725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:33.947519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:33.983796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce9cb2-e044-42a2-b4ef-f54605bec215 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:34.007553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:34.009519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:34.010126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:34.010564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] PATH_INFO: `/v3/users/6785df5e54c548c6a6a42137c6c412ae` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:34.012640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6785df5e54c548c6a6a42137c6c412ae)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:34.015911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:34.080334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c8b31a3d3e45e1b87985d948cec440', 'request_id': 'req-68f69d5e-9828-49af-96e5-370ee04a0213'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.091246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c8b31a3d3e45e1b87985d948cec440', 'request_id': 'req-68f69d5e-9828-49af-96e5-370ee04a0213'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.148536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:34.244057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c8b31a3d3e45e1b87985d948cec440', 'request_id': 'req-68f69d5e-9828-49af-96e5-370ee04a0213'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.257109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] The token cache is being invalidate because project 55c8b31a3d3e45e1b87985d948cec440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:34.257413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.258118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.262586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 842/1680] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:33 2022] DELETE /identity/v3/projects/55c8b31a3d3e45e1b87985d948cec440 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:34.262586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:34.262586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:34.313978 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6785df5e54c548c6a6a42137c6c412ae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.339296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6785df5e54c548c6a6a42137c6c412ae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.340869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6785df5e54c548c6a6a42137c6c412ae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.361139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:34.375931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6785df5e54c548c6a6a42137c6c412ae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.464506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69d5e-9828-49af-96e5-370ee04a0213 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:34.478554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6785df5e54c548c6a6a42137c6c412ae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:34.479820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:34.480943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:34.481425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:34.481714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:34.484937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:34.487909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 839/1681] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:33 2022] DELETE /identity/v3/users/6785df5e54c548c6a6a42137c6c412ae => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:34.489146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:34.491754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:34.500198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:34.502334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:34.515475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 843/1682] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:34.516942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:34.524268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:34.689197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd423f9c-f4a1-4f1a-96e5-1d95e682df74 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:34.716256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:34.716936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:34.717377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:34.717823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:34.718660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:34.724007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:34.807295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481307e9-4962-420b-942d-00977ed877e8 service placement] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:34.835921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:34.837236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:34.837236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:34.837513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] PATH_INFO: `/v3/users/5c8dac68c90544159d4670fa061006a9` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:34.839276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8dac68c90544159d4670fa061006a9)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:34.841544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:34.852660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 840/1683] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:34 2022] POST /identity/v3/projects => generated 381 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:34.856720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:35.010185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d4b4f-6a2f-4ea9-b3bc-8676a1e7ed72 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:35.045823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:35.046530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:35.046817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:35.047095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:35.047842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:35.049789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:35.125641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8dac68c90544159d4670fa061006a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:35.126360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8dac68c90544159d4670fa061006a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:35.162127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8dac68c90544159d4670fa061006a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:35.167456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8dac68c90544159d4670fa061006a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:35.219597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 841/1684] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:34 2022] POST /identity/v3/users => generated 297 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:27:35.223567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:35.228817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:35.238419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8dac68c90544159d4670fa061006a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:35.247231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 844/1685] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:34 2022] DELETE /identity/v3/users/5c8dac68c90544159d4670fa061006a9 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:35.302802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7baeee-d34d-4933-9992-f0a97034b586 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:35.480931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:35.481944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:35.482842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:35.484669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:35.485760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:35.491090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:35.521506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 842/1686] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:35.525850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:35.545470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e19371-46a7-406f-b916-74170f72c280 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:35.569459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:35.575849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:35.580194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:35.580194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] PATH_INFO: `/v3/projects/91c16b8d08294f3c958f3657bbc4e6a3/users/bf786646f4694be0963fe4f6077d0f7c/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:35.603003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c16b8d08294f3c958f3657bbc4e6a3, user_id=bf786646f4694be0963fe4f6077d0f7c, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:35.607273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:35.696111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 845/1687] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:35 2022] PUT /identity/v3/projects/91c16b8d08294f3c958f3657bbc4e6a3/users/bf786646f4694be0963fe4f6077d0f7c/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:35.697948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:35.718830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def3cff-8cbc-478b-8070-6aafdf0ca494 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:35.854453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:35.858137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:35.865478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:35.867737 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:35.868026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:35.868437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:35.869196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:35.870828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:35.890505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcc418-944d-4f5f-a611-770772822241 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:35.898686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:35.900588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:35.901030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:35.901456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] PATH_INFO: `/v3/projects/eb47091723aa4c1db05243cdf458e529` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:35.903078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb47091723aa4c1db05243cdf458e529)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:35.916138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 843/1688] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:35.917483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:35.920614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:35.946427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b7292-2f62-486a-aba9-f7d6f6f7d6ad admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:35.959067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a80cb-228a-43dd-b893-d1165cc4b87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:35.961049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a80cb-228a-43dd-b893-d1165cc4b87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:35.961537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a80cb-228a-43dd-b893-d1165cc4b87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:35.962062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a80cb-228a-43dd-b893-d1165cc4b87e admin admin] PATH_INFO: `/v3/projects/91c16b8d08294f3c958f3657bbc4e6a3/users/bf786646f4694be0963fe4f6077d0f7c/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:35.968630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a80cb-228a-43dd-b893-d1165cc4b87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c16b8d08294f3c958f3657bbc4e6a3, user_id=bf786646f4694be0963fe4f6077d0f7c, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:35.974010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a80cb-228a-43dd-b893-d1165cc4b87e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:35.990943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 844/1689] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:35 2022] PUT /identity/v3/projects/91c16b8d08294f3c958f3657bbc4e6a3/users/bf786646f4694be0963fe4f6077d0f7c/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:36.004972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:36.005574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:36.006120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:36.080876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] MFA Rules not processed for user `bf786646f4694be0963fe4f6077d0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:36.082978 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:36.204844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb47091723aa4c1db05243cdf458e529', 'request_id': 'req-860617ba-5b76-46ea-ace5-989a24d358b6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:36.248733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb47091723aa4c1db05243cdf458e529', 'request_id': 'req-860617ba-5b76-46ea-ace5-989a24d358b6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:36.270951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 845/1690] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:36.288034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:36.289598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:36.377838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb47091723aa4c1db05243cdf458e529', 'request_id': 'req-860617ba-5b76-46ea-ace5-989a24d358b6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:36.395267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1c526-7522-4625-be67-396482a3fd58 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:36.413646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] The token cache is being invalidate because project eb47091723aa4c1db05243cdf458e529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:36.415073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:36.417320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-860617ba-5b76-46ea-ace5-989a24d358b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:36.425294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 846/1691] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:35 2022] DELETE /identity/v3/projects/eb47091723aa4c1db05243cdf458e529 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:36.431264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9494e-9ce4-48e9-9ee4-bc980cc8eade None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:36.431861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9494e-9ce4-48e9-9ee4-bc980cc8eade None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:36.432338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9494e-9ce4-48e9-9ee4-bc980cc8eade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:36.476958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-18b9494e-9ce4-48e9-9ee4-bc980cc8eade None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:36.482159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-18b9494e-9ce4-48e9-9ee4-bc980cc8eade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:36.519633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:36.520301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:36.521018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:36.521018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:36.529795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:36.620747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 847/1692] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:36.622918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:36.622918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:36.623099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:36.661299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:36.662542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:36.691222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 848/1693] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:36.692344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:36.705384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:36.705874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:36.706991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f07b51-f922-4c38-a7b7-1833618b7a87 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:36.712577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:36.713599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:36.716305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:36.716859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:36.717864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:36.718376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:36.720385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:36.731727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 846/1694] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:27:36.771131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:36.771425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 849/1695] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:36.796234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b72816-5f84-405a-8322-d0975744f5fd service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:36.801633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:36.802670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:36.803128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:36.803604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:36.804522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:36.806778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:36.847530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:36.850828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:36.909272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 847/1696] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:36 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:36.912350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:36.917409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813b91-c8aa-435c-90ca-069192acb346 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:36.931606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:37.094961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71822ea-2aa7-4e50-830b-44813b30d814 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:37.120486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:37.121708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.122908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.122908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.123112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:37.125066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:37.157868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:37.163469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.165904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.166392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] PATH_INFO: `/v3/projects/8e17a41e261f40088f9ae753780e6f01` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.169956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e17a41e261f40088f9ae753780e6f01)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:37.172198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:37.235003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:37.282066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 848/1697] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:36 2022] POST /identity/v3/users => generated 288 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:37.284154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:37.287796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:37.343951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e17a41e261f40088f9ae753780e6f01', 'request_id': 'req-d53a4228-6eb5-4905-82cc-9b760056156f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:37.362414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e17a41e261f40088f9ae753780e6f01', 'request_id': 'req-d53a4228-6eb5-4905-82cc-9b760056156f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:37.447798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3e2fb-ba07-471c-8cfe-a0615e3ac1fc admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:37.448646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e17a41e261f40088f9ae753780e6f01', 'request_id': 'req-d53a4228-6eb5-4905-82cc-9b760056156f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:37.483671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] The token cache is being invalidate because project 8e17a41e261f40088f9ae753780e6f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:37.483975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:37.489594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:37.494394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 850/1698] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:36 2022] DELETE /identity/v3/projects/8e17a41e261f40088f9ae753780e6f01 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:37.496428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:37.502163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:37.541336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:37.542010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.544817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.544817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.544817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:37.612012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:37.614346 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc355557-b102-4999-bf74-2d0d9f034c33 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:37.628321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 849/1699] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:37.629864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8018079-4a67-4681-bba0-280159768d25 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.630157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8018079-4a67-4681-bba0-280159768d25 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.630480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8018079-4a67-4681-bba0-280159768d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.655673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d8018079-4a67-4681-bba0-280159768d25 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:37.657058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d8018079-4a67-4681-bba0-280159768d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:37.694388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4228-6eb5-4905-82cc-9b760056156f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:37.707527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:37.708247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.708539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.708826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.709634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:37.717803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:37.760793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 851/1700] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:37.767727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:37.791889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 850/1701] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:37.793442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a20b99-991a-43b6-a230-185de2986c11 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.793733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a20b99-991a-43b6-a230-185de2986c11 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.794016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a20b99-991a-43b6-a230-185de2986c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.795193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef49c-10c3-4644-9ad0-f33173bdf4ba admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:37.809846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:37.810585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.810881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.811151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.811961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:37.815832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:37.836359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-86a20b99-991a-43b6-a230-185de2986c11 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:37.837622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-86a20b99-991a-43b6-a230-185de2986c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:37.841147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 852/1702] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:27:37.842885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:37.860945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 851/1703] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:37.866514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb7db3-fbd8-41ca-bd80-f8590abe7f10 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:37.873354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:37.874286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.874636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.874926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] PATH_INFO: `/v3/projects/1f18c992938d4a99beef4376d6a68565/users/fd9054bbfbc44d4e9bf64686dfcd21c3/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.877738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576b096-70b8-4c6a-8558-88407742e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:37.878262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576b096-70b8-4c6a-8558-88407742e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:37.878764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576b096-70b8-4c6a-8558-88407742e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:37.898963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f18c992938d4a99beef4376d6a68565, user_id=fd9054bbfbc44d4e9bf64686dfcd21c3, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:37.901893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:37.928423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-8576b096-70b8-4c6a-8558-88407742e7e1 None None] MFA Rules not processed for user `85df2f6595d3489bb5d1d547cb2cd900`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:37.929651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8576b096-70b8-4c6a-8558-88407742e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:38.021784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 853/1704] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:37 2022] PUT /identity/v3/projects/1f18c992938d4a99beef4376d6a68565/users/fd9054bbfbc44d4e9bf64686dfcd21c3/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:38.022875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.033604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 852/1705] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:38.035513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400d78-fe75-41c7-94b8-28e4011fc662 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.036398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400d78-fe75-41c7-94b8-28e4011fc662 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.040627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400d78-fe75-41c7-94b8-28e4011fc662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.093234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b9400d78-fe75-41c7-94b8-28e4011fc662 None None] MFA Rules not processed for user `12816eca659542af9e41b92fdb674eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:38.094750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b9400d78-fe75-41c7-94b8-28e4011fc662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:38.163612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247a681-3562-45f2-9bba-27f493a356cc admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.168524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.169186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.169496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.169776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.170521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:38.170828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.176564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:38.198707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 853/1706] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:38.200600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.202021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.202307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.211511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 854/1707] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:38.211511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.212969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:38.266297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] MFA Rules not processed for user `fd9054bbfbc44d4e9bf64686dfcd21c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:38.269722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:38.373323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 854/1708] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:38.374225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.374330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e7374-2a14-4274-ac46-9f30882ff0da admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.392119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6fb24-c56b-484b-8c90-5356b9d6ce9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.393129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fb24-c56b-484b-8c90-5356b9d6ce9f service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.394759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fb24-c56b-484b-8c90-5356b9d6ce9f service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.395119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fb24-c56b-484b-8c90-5356b9d6ce9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.400548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f146-6b8e-4bc7-8b64-96d8d77d5b43 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.409164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.409932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.410254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.410531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.411545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.414090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:38.457903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:38.466685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6fb24-c56b-484b-8c90-5356b9d6ce9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.468450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6fb24-c56b-484b-8c90-5356b9d6ce9f service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:38.478299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 855/1709] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:38.479484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.479816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.480055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.504592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] MFA Rules not processed for user `12816eca659542af9e41b92fdb674eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:38.506068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:38.515666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 856/1710] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:38.516614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.530761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e833d6-c1b8-4534-be15-956f3e694977 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.531514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 855/1711] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:38.532934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.537272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:38.641214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.641908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.642196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.642539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.643730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.645623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:38.709486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:38.712974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a4a20-5ccd-4942-ac40-8754db1a456e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.718865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.719679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.719956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.720236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.786001 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.787740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:38.799031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 857/1712] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/projects => generated 377 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:38.799031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.827747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 856/1713] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:38.828736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:38.854068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440db77d-b8c5-4c31-88dc-95aed31a1c99 service nova] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.940548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1f14a-8ec9-4c60-be2b-78c2b309490e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:38.954092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.954800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.955092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.955584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.956429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.959426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:38.985669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:38.986639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:38.986928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:38.987204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:38.988106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:38.990327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.063511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 858/1714] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/users => generated 303 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:39.067681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.091126 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16713c0a-2056-44ac-bb2a-08796a5104ff admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.104139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.108547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.108547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.108547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.108547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.111141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.125498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 859/1715] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:39.141529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 857/1716] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:38 2022] POST /identity/v3/users => generated 295 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:39.141672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.142678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.154314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4657707-f3ca-463b-b295-9c475f8c7a7c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.162341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.163283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.163885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.164376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.165353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.166135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951c93-bacd-4868-a6b1-d110c9d2b8bc admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.169765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.181704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 860/1717] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:39.185392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.191583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.192296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.192589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.192876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.193652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.195344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.199200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a367c-1452-4f44-83ce-bc510a3baf96 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.206412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa5e9b-ae12-4682-828b-81eda2fd0aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.208160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5e9b-ae12-4682-828b-81eda2fd0aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.208555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5e9b-ae12-4682-828b-81eda2fd0aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.208935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5e9b-ae12-4682-828b-81eda2fd0aae admin admin] PATH_INFO: `/v3/projects/d3671fb4f2a04b328823f20a463b4058/users/d41ae6efd69248648c95d5fae255118e/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.226694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 858/1718] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:39.228064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.234436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa5e9b-ae12-4682-828b-81eda2fd0aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3671fb4f2a04b328823f20a463b4058, user_id=d41ae6efd69248648c95d5fae255118e, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.240241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa5e9b-ae12-4682-828b-81eda2fd0aae admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.250688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8867e3-d85e-41be-8f86-cce0b2102107 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.259531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.261411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.262521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.263187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] PATH_INFO: `/v3/projects/3599d124c77e4ab295d47ae98bb572fe/users/babf4e70d3254767ae6288d35bca72e8/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.284906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3599d124c77e4ab295d47ae98bb572fe, user_id=babf4e70d3254767ae6288d35bca72e8, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.288969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.336209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 861/1719] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:39 2022] PUT /identity/v3/projects/d3671fb4f2a04b328823f20a463b4058/users/d41ae6efd69248648c95d5fae255118e/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:39.338496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.339216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.339659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.358320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 859/1720] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:39 2022] PUT /identity/v3/projects/3599d124c77e4ab295d47ae98bb572fe/users/babf4e70d3254767ae6288d35bca72e8/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:39.368264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.380223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:39.382166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:39.505894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ea0df-f079-49b3-a0ea-dca5bb8e1744 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.514143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.515379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.515869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.517014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.520647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.522789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.544209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 860/1721] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:39.548267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.575083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 862/1722] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:39.612229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370279b-a565-4ac6-9468-ffc854f91aa3 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.617272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:39.619157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:39.648730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415052d0-738e-4964-bcff-6de2b3afad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.650784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415052d0-738e-4964-bcff-6de2b3afad0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.651677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415052d0-738e-4964-bcff-6de2b3afad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.652146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415052d0-738e-4964-bcff-6de2b3afad0d admin admin] PATH_INFO: `/v3/projects/3599d124c77e4ab295d47ae98bb572fe/users/babf4e70d3254767ae6288d35bca72e8/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.655769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415052d0-738e-4964-bcff-6de2b3afad0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3599d124c77e4ab295d47ae98bb572fe, user_id=babf4e70d3254767ae6288d35bca72e8, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:39.660408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415052d0-738e-4964-bcff-6de2b3afad0d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:39.677191 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 861/1723] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:39 2022] PUT /identity/v3/projects/3599d124c77e4ab295d47ae98bb572fe/users/babf4e70d3254767ae6288d35bca72e8/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:39.678734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e801bd-e9d5-4cc0-8800-6782aa0c36bd None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.679369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e801bd-e9d5-4cc0-8800-6782aa0c36bd None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.679811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e801bd-e9d5-4cc0-8800-6782aa0c36bd None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.701063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 862/1724] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jun 1 04:27:39 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:39.702773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af013d-529c-46bc-b55f-3801aac3e965 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.703897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af013d-529c-46bc-b55f-3801aac3e965 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.704241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af013d-529c-46bc-b55f-3801aac3e965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.747228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-52af013d-529c-46bc-b55f-3801aac3e965 None None] MFA Rules not processed for user `babf4e70d3254767ae6288d35bca72e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:39.748698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-52af013d-529c-46bc-b55f-3801aac3e965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:39.767766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff98250-3cb7-4c14-abf5-fda6f1c87be2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:39.857051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:39.896784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:39.897794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.898506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.898931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:39.903338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:39.974253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 863/1725] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:39.975961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:39.976651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:39.977061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:40.037060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:40.038478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:40.047032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:40.048655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:40.055605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 864/1726] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:27:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:40.056591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:40.058259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:40.078405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 863/1727] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:40.079388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:40.086048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:40.112334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b1164-4ec5-4c87-9271-939fecf7e15b service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:40.125862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25bf1-d15f-4a82-97c5-aae83bdb19ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:40.126810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25bf1-d15f-4a82-97c5-aae83bdb19ff service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:40.127098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25bf1-d15f-4a82-97c5-aae83bdb19ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:40.127399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25bf1-d15f-4a82-97c5-aae83bdb19ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:40.147938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25bf1-d15f-4a82-97c5-aae83bdb19ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:40.149601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25bf1-d15f-4a82-97c5-aae83bdb19ff service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:40.151884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:40.183106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e448818-75f6-4b41-adc1-bcda6d2ccdeb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:40.190898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 864/1728] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:40.191977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0bdb-d87e-4857-9cf4-4e5765d9831e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:40.193024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0bdb-d87e-4857-9cf4-4e5765d9831e service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:40.194124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0bdb-d87e-4857-9cf4-4e5765d9831e service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:40.194606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0bdb-d87e-4857-9cf4-4e5765d9831e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:40.388298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0bdb-d87e-4857-9cf4-4e5765d9831e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:40.389964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0bdb-d87e-4857-9cf4-4e5765d9831e service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:40.430789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 865/1729] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:40.622315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:40.686501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:40.927419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:41.422538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:41.423122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:41.423122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:41.480298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] MFA Rules not processed for user `12816eca659542af9e41b92fdb674eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:41.485282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:41.580230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:41.772293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 865/1730] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:42.173787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:42.271252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bbb46-b519-4ff7-8fbf-c8b034b80cfd None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:42.285444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bbb46-b519-4ff7-8fbf-c8b034b80cfd None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:42.285444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bbb46-b519-4ff7-8fbf-c8b034b80cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:42.319531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-606bbb46-b519-4ff7-8fbf-c8b034b80cfd None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:42.321642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-606bbb46-b519-4ff7-8fbf-c8b034b80cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:42.436276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 866/1731] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:27:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:42.926046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:42.927543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:43.077097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ff94c-5691-475f-b66b-1b50f826de42 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:43.079717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8909cf-f079-4764-a25c-3f11bc50af46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:43.081436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8909cf-f079-4764-a25c-3f11bc50af46 service swift] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:43.081886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8909cf-f079-4764-a25c-3f11bc50af46 service swift] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:43.082318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8909cf-f079-4764-a25c-3f11bc50af46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:43.094670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8909cf-f079-4764-a25c-3f11bc50af46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:43.098162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8909cf-f079-4764-a25c-3f11bc50af46 service swift] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:43.111419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 866/1732] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:27:43.144622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:43.967692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:43.968255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:43.968255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:43.979397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:44.075969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:44.095132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:44.132197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 867/1733] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Jun 1 04:27:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:44.200180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:44.200689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:44.200689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:44.268020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] MFA Rules not processed for user `d41ae6efd69248648c95d5fae255118e`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:44.273227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:44.509863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:44.513083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 867/1734] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:44.513265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3032c-96a1-4779-bb1a-247dd9b221a5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:44.513265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca61e44-3e49-4150-ac68-3bb7835de615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:44.513265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca61e44-3e49-4150-ac68-3bb7835de615 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:44.513265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca61e44-3e49-4150-ac68-3bb7835de615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:44.513612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca61e44-3e49-4150-ac68-3bb7835de615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:44.513686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:44.515844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca61e44-3e49-4150-ac68-3bb7835de615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:44.518265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca61e44-3e49-4150-ac68-3bb7835de615 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:44.693993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 868/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:44.696881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9893d-465d-4b7d-ba1a-a8d0aae566cd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:44.696881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:44.696881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:44.696881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:44.699171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:44.699171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:44.699171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:45.019682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:45.020652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:45.021002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:45.056010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] MFA Rules not processed for user `85df2f6595d3489bb5d1d547cb2cd900`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:45.056010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:45.089912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 868/1736] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:44 2022] POST /identity/v3/projects => generated 405 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:27:45.092112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:45.178899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:45.296460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ea6dd-33e7-44af-98bf-7e08f680dff6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:45.304329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:45.305440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:45.305945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:45.312523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:45.312523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:45.312523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:45.317048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 869/1737] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:45.436411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 869/1738] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:45 2022] POST /identity/v3/users => generated 309 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:27:45.440468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:45.485843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d852c-a5cb-43db-a352-daf2811c459b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:45.531980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:45.534557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:45.537015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:45.537708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:45.541076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:45.565553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:45.609203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 870/1739] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:45.613547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:45.631273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446f232-5908-48fb-8ff9-700ec6b66924 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:45.658738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:45.662360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:45.662770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:45.663243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] PATH_INFO: `/v3/projects/95308535343e449b81d3462990cbfbf6/users/c8928cdcfe914fa8bc466e52d85f9b04/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:45.727418 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95308535343e449b81d3462990cbfbf6, user_id=c8928cdcfe914fa8bc466e52d85f9b04, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:45.742663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:45.851919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 870/1740] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:45 2022] PUT /identity/v3/projects/95308535343e449b81d3462990cbfbf6/users/c8928cdcfe914fa8bc466e52d85f9b04/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:45.856325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:45.897119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda66c-fa4b-4f86-abe0-4dcbdd457543 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:46.098832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:46.100412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.100925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.101408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:46.102358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:46.109198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:46.132078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 871/1741] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:46.136399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:46.150892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09d0b-c144-4f1f-b474-eae119dead9d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:46.163106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bd289-4b95-46a4-ad93-287ab6779068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:46.164198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bd289-4b95-46a4-ad93-287ab6779068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.164597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bd289-4b95-46a4-ad93-287ab6779068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.164995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bd289-4b95-46a4-ad93-287ab6779068 admin admin] PATH_INFO: `/v3/projects/95308535343e449b81d3462990cbfbf6/users/c8928cdcfe914fa8bc466e52d85f9b04/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:46.169475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bd289-4b95-46a4-ad93-287ab6779068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95308535343e449b81d3462990cbfbf6, user_id=c8928cdcfe914fa8bc466e52d85f9b04, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:46.173114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bd289-4b95-46a4-ad93-287ab6779068 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:46.205394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 871/1742] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:46 2022] PUT /identity/v3/projects/95308535343e449b81d3462990cbfbf6/users/c8928cdcfe914fa8bc466e52d85f9b04/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:46.369310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:46.371815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:46.402183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8df0c9-c499-48b9-ae48-fcc675d29053 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:46.511473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cea65-d5e7-43c5-94dd-cb6bb71efac7 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.512700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cea65-d5e7-43c5-94dd-cb6bb71efac7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.513195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cea65-d5e7-43c5-94dd-cb6bb71efac7 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:46.515120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 872/1743] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:27:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:46.532648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:46.534764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.535056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.535457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] PATH_INFO: `/v3/users/4801021c8f4741d9952c5db2f0e22f1a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:46.536771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4801021c8f4741d9952c5db2f0e22f1a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:46.538807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:46.574264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.575536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.576164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:46.580038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:46.610078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:46.616502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:46.804817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 873/1744] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:27:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:46.825448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:46.826310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4801021c8f4741d9952c5db2f0e22f1a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:46.835688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4801021c8f4741d9952c5db2f0e22f1a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:46.837229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4801021c8f4741d9952c5db2f0e22f1a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:46.853686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4801021c8f4741d9952c5db2f0e22f1a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:46.854318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc80ba3-d09f-4a94-9f6a-8e5fdbdefb95 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:46.863156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:46.863862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.866393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.866983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:46.869258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:46.894008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:46.895945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba5e15-bd57-4e5b-a12f-4530ff3bf302 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:46.914788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 874/1745] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:27:46.943528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4801021c8f4741d9952c5db2f0e22f1a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:46.951865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 872/1746] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:46 2022] DELETE /identity/v3/users/4801021c8f4741d9952c5db2f0e22f1a => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:46.964569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:46.965544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:46.966119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:47.024169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:47.025738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:47.134368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:47.151171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 875/1747] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:47.185542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:47.188013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:47.290870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7369d9-f874-4e9e-b6cb-c7ea306e96f7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:47.312090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb9810-3580-4023-b822-5b2df0d9dd63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:47.312887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9810-3580-4023-b822-5b2df0d9dd63 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:47.313186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9810-3580-4023-b822-5b2df0d9dd63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:47.313473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9810-3580-4023-b822-5b2df0d9dd63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:47.351201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb9810-3580-4023-b822-5b2df0d9dd63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:47.353476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb9810-3580-4023-b822-5b2df0d9dd63 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:47.368337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 873/1748] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:47.867495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:47.867495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:47.883074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837968c3-30ae-4d62-bf81-b71c253b25f7 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:47.890574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:47.891393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:47.891716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:47.892076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] PATH_INFO: `/v3/projects/7a45f5b11ffc4a99b4d7d7b8fbf86838` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:47.894009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a45f5b11ffc4a99b4d7d7b8fbf86838)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:47.896960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:48.057506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a45f5b11ffc4a99b4d7d7b8fbf86838', 'request_id': 'req-e109b869-1207-4446-95b2-c93bf35acb29'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:48.064536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a45f5b11ffc4a99b4d7d7b8fbf86838', 'request_id': 'req-e109b869-1207-4446-95b2-c93bf35acb29'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:48.160202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a45f5b11ffc4a99b4d7d7b8fbf86838', 'request_id': 'req-e109b869-1207-4446-95b2-c93bf35acb29'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:48.174570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] The token cache is being invalidate because project 7a45f5b11ffc4a99b4d7d7b8fbf86838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:48.174869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:48.175579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e109b869-1207-4446-95b2-c93bf35acb29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:48.177246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 876/1749] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:47 2022] DELETE /identity/v3/projects/7a45f5b11ffc4a99b4d7d7b8fbf86838 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:48.182443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e218-8351-4403-a6cf-73fe14cf2844 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.182739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e218-8351-4403-a6cf-73fe14cf2844 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.183038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99e218-8351-4403-a6cf-73fe14cf2844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.202906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-da99e218-8351-4403-a6cf-73fe14cf2844 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:48.204161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-da99e218-8351-4403-a6cf-73fe14cf2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:48.336068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.336369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.336562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 874/1750] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:48.336656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.342980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.344087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.344087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.404632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:48.409569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:48.427074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] MFA Rules not processed for user `c8928cdcfe914fa8bc466e52d85f9b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:48.429291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:48.436676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 875/1751] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:48.438902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:48.452665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fb3c3-8fee-42e8-8a82-b84106560f96 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:48.459040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:48.460077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.460721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.461356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.462251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:48.462712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:48.465508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:48.480435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 876/1752] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:48.485894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:48.503414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4700d-5777-4e10-bbfd-2fe66cf5abed admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:48.514455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:48.515917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.517175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.517944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.520080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:48.524342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:48.555127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:48.561719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 877/1753] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:48 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:27:48.651783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:48.652073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 877/1754] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:48 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:48.673035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda06f8a-b6e4-4239-b001-654050fd2c7d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:48.800434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:48.803069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.803414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.803688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.804755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:48.807030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:48.818199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:48.904000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 878/1755] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:48 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:27:48.908566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:48.943514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd2ca6-4824-4b2c-a7c5-40b6be95d148 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:48.950229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:48.951190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.951841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.952320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.953793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:48.957196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:48.974223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 878/1756] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:48.976038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:48.987903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f39d3-f0c2-4f60-a89d-608648b89eae admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:48.994240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1603a-8077-4fba-846a-371d7e677382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:48.995118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1603a-8077-4fba-846a-371d7e677382 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:48.995615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1603a-8077-4fba-846a-371d7e677382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:48.996447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1603a-8077-4fba-846a-371d7e677382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:48.997348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1603a-8077-4fba-846a-371d7e677382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:48.999136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1603a-8077-4fba-846a-371d7e677382 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:49.018102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 879/1757] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:49.019374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:49.030944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663e5f3-d264-4e74-9cb4-fdde0e3d2b7f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:49.036267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:49.037191 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:49.037526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:49.037835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] PATH_INFO: `/v3/projects/026a8415ad6e473aadbeb43ce073f9a0/users/b51a244ac5de4191b282af104758b6b8/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:49.054195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=026a8415ad6e473aadbeb43ce073f9a0, user_id=b51a244ac5de4191b282af104758b6b8, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:49.057021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:49.161386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 879/1758] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:49 2022] PUT /identity/v3/projects/026a8415ad6e473aadbeb43ce073f9a0/users/b51a244ac5de4191b282af104758b6b8/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:49.174686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44620b3-f807-4f27-a641-934157acd393 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:49.175030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44620b3-f807-4f27-a641-934157acd393 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:49.175327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44620b3-f807-4f27-a641-934157acd393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:49.203951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f44620b3-f807-4f27-a641-934157acd393 None None] MFA Rules not processed for user `b51a244ac5de4191b282af104758b6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:49.205245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f44620b3-f807-4f27-a641-934157acd393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:49.273054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 880/1759] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:49.283825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:49.285576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:49.344743 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796d065-1d50-42b3-baea-8e4323606836 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:49.349056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8a4d3-9179-4110-b3c2-3f2c5541b221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:49.349874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a4d3-9179-4110-b3c2-3f2c5541b221 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:49.350230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a4d3-9179-4110-b3c2-3f2c5541b221 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:49.350550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8a4d3-9179-4110-b3c2-3f2c5541b221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:49.360226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8a4d3-9179-4110-b3c2-3f2c5541b221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:49.362918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8a4d3-9179-4110-b3c2-3f2c5541b221 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:49.384697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 880/1760] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:49.537425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415946d9-17cc-4a10-857d-6408a7b5002e None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:49.537849 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415946d9-17cc-4a10-857d-6408a7b5002e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:49.538458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415946d9-17cc-4a10-857d-6408a7b5002e None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:49.540643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 881/1761] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:49.565905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e7011-2977-4b94-b3b9-b701dee7a91e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:49.566208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e7011-2977-4b94-b3b9-b701dee7a91e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:49.567004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e7011-2977-4b94-b3b9-b701dee7a91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:49.598298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4c5e7011-2977-4b94-b3b9-b701dee7a91e None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:49.600620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4c5e7011-2977-4b94-b3b9-b701dee7a91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:49.700919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 881/1762] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:49.712962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415946d9-17cc-4a10-857d-6408a7b5002e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:49.717813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-415946d9-17cc-4a10-857d-6408a7b5002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:49.744695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415946d9-17cc-4a10-857d-6408a7b5002e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:49.760182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd09228-ed53-4fdd-818b-304e900b6231 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:49.761360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd09228-ed53-4fdd-818b-304e900b6231 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:49.761898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd09228-ed53-4fdd-818b-304e900b6231 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:49.762524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd09228-ed53-4fdd-818b-304e900b6231 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:49.775221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd09228-ed53-4fdd-818b-304e900b6231 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:49.777218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd09228-ed53-4fdd-818b-304e900b6231 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:49.810977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 882/1763] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:50.885949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470c9a-9bd2-4b69-b823-31e236ab7d3c None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:50.887512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470c9a-9bd2-4b69-b823-31e236ab7d3c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:50.888086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79470c9a-9bd2-4b69-b823-31e236ab7d3c None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:50.889986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 882/1764] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:27:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:27:50.921075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:50.921627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:50.921777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:50.943647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] MFA Rules not processed for user `6848ac67d15b406693e33c9dfabbbe89`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:50.944991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:50.953994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 883/1765] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:27:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:50.965028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79470c9a-9bd2-4b69-b823-31e236ab7d3c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:50.978860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79470c9a-9bd2-4b69-b823-31e236ab7d3c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:50.984594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:50.985228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:50.985543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:50.985790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:50.994516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:50.996795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:51.011440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 883/1766] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:51.044612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:51.046287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:51.117677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0386d-0076-4f1c-88f5-02a525096b68 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:51.123618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:51.124688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:51.124993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:51.125372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] PATH_INFO: `/v3/users/d41ae6efd69248648c95d5fae255118e` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:51.126629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41ae6efd69248648c95d5fae255118e)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:51.128636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:51.292089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41ae6efd69248648c95d5fae255118e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:51.292089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41ae6efd69248648c95d5fae255118e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:51.292089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=1,keystone:SELECT=91,keystone:DELETE=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:51.299996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41ae6efd69248648c95d5fae255118e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:51.306778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41ae6efd69248648c95d5fae255118e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:51.354413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41ae6efd69248648c95d5fae255118e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:51.365547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 884/1767] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:51 2022] DELETE /identity/v3/users/d41ae6efd69248648c95d5fae255118e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:51.640313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:51.651651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8023c42-ff4e-4546-98f3-990f2b5cb2a4 service placement] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:51.697994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:51.710774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:51.711671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:51.711949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:51.712250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] PATH_INFO: `/v3/projects/d3671fb4f2a04b328823f20a463b4058` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:51.713443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3671fb4f2a04b328823f20a463b4058)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:51.716127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:51.899022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3671fb4f2a04b328823f20a463b4058', 'request_id': 'req-8f7e3695-8692-490e-8f19-1db9a8696de9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.022817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3671fb4f2a04b328823f20a463b4058', 'request_id': 'req-8f7e3695-8692-490e-8f19-1db9a8696de9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.029628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3671fb4f2a04b328823f20a463b4058', 'request_id': 'req-8f7e3695-8692-490e-8f19-1db9a8696de9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.041012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] The token cache is being invalidate because project d3671fb4f2a04b328823f20a463b4058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:52.041302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.041966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.043606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 884/1768] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:51 2022] DELETE /identity/v3/projects/d3671fb4f2a04b328823f20a463b4058 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:52.047742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:52.050043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:52.064813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832b61c-ec2a-497b-8d07-bfd3122a991d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:52.127698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:52.128588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:52.128873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:52.129152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] PATH_INFO: `/v3/users/c8928cdcfe914fa8bc466e52d85f9b04` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:52.130414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8928cdcfe914fa8bc466e52d85f9b04)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:52.135828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:52.339741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8928cdcfe914fa8bc466e52d85f9b04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.341294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8928cdcfe914fa8bc466e52d85f9b04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.372382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8928cdcfe914fa8bc466e52d85f9b04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.380903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8928cdcfe914fa8bc466e52d85f9b04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.457477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8928cdcfe914fa8bc466e52d85f9b04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:52.466502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 885/1769] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:52 2022] DELETE /identity/v3/users/c8928cdcfe914fa8bc466e52d85f9b04 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:52.786747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:52.788440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:52.814404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3695-8692-490e-8f19-1db9a8696de9 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:52.942814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:52.946867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:52.946867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:52.946867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] PATH_INFO: `/v3/projects/95308535343e449b81d3462990cbfbf6` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:52.946867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=95308535343e449b81d3462990cbfbf6)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:52.947793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:52.948450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:52.950029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:53.019173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0b5aa-62b7-4bdf-966e-0c8de7e16dba admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.023896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.024553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.027958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.027958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.027958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:53.039055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:53.112678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.114066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e24fc2-59b6-44ea-8ab3-e33e224cb56c service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.123951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 886/1770] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:53.157826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95308535343e449b81d3462990cbfbf6', 'request_id': 'req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:53.215942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95308535343e449b81d3462990cbfbf6', 'request_id': 'req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:53.222055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95308535343e449b81d3462990cbfbf6', 'request_id': 'req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:53.234037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] The token cache is being invalidate because project 95308535343e449b81d3462990cbfbf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:53.234337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:53.234991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-387c0efa-84cc-45f5-b81a-ccb44c7dfd4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:53.236553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 885/1771] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:52 2022] DELETE /identity/v3/projects/95308535343e449b81d3462990cbfbf6 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:53.241314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.241618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.241901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.267866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:53.269139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:53.339209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 887/1772] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:53.347523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.347990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.352740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.362338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:53.368260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:53.369542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:53.377646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 886/1773] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:53.381665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:53.392775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be793e-9e7f-491b-add0-1614fa831eb0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.397389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.398046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.398335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.398658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.399423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:53.399738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.401352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.414392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 888/1774] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:53.418861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:53.428072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8976587-ab62-4309-86f1-62dca1443577 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.432806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.433455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.433994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.434070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.434746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.436689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.572176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 887/1775] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:53 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:27:53.574109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:53.588234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d3d9a-6809-46a5-bd11-930bed478562 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.678484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.680360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.680639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.680935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.681626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.684175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.783685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 889/1776] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:53 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:53.787384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:53.798334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14f9b0-689e-4b16-8993-a99473a13731 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.803077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.804028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.804386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.804675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.805419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.807033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.818876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 888/1777] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:53.823093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:53.834734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe50e7b-5644-44c2-80ea-70adde7e4149 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.839950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd369a09-3949-4dd1-a7bc-f895e2ed3e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.840623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd369a09-3949-4dd1-a7bc-f895e2ed3e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.840906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd369a09-3949-4dd1-a7bc-f895e2ed3e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.841178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd369a09-3949-4dd1-a7bc-f895e2ed3e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.841896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd369a09-3949-4dd1-a7bc-f895e2ed3e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.843650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd369a09-3949-4dd1-a7bc-f895e2ed3e4b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.865313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 890/1778] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:53.870040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:53.881988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca2894-eabb-413f-b9de-a9b765b7d5d3 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:53.888168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:53.894269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.894269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.894269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] PATH_INFO: `/v3/projects/244e62e9b8664e4b95ff54af08a2128e/users/8449bd44ea5a4c1f899576810e63470f/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:53.905038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244e62e9b8664e4b95ff54af08a2128e, user_id=8449bd44ea5a4c1f899576810e63470f, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:53.907753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:53.985994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 889/1779] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:53 2022] PUT /identity/v3/projects/244e62e9b8664e4b95ff54af08a2128e/users/8449bd44ea5a4c1f899576810e63470f/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:53.998906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:53.999196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:53.999512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.012810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:54.028015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] MFA Rules not processed for user `8449bd44ea5a4c1f899576810e63470f`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:54.029253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:54.098947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 891/1780] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:54.103473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.176949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0603-f9e2-4343-a1fe-b59d771ea169 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:54.182700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:54.183410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.184357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.184904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.185316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:54.187321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:54.300522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 890/1781] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:54 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:54.302610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.314255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e032b-f8d1-4a95-a796-e0ee26846984 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:54.377451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:54.378141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.378430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.378709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.379459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:54.381453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:54.460838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 892/1782] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:27:54 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:54.462285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.473560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148663f-6deb-42b1-bdb3-13d74a9654a8 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:54.478651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:54.479370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.479663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.479950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.481443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:54.483367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:54.498803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 891/1783] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:54.501028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.516971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8c844-5c73-4e76-b45a-49c04a4947c8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:54.523084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:54.524139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.524489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.526369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] PATH_INFO: `/v3/projects/6eb455ba7c994dcaa019094234bc796c/users/af0d167c5ac14b84bac501684ea65a9d/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.544966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb455ba7c994dcaa019094234bc796c, user_id=af0d167c5ac14b84bac501684ea65a9d, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:54.547858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:54.664770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 893/1784] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:54 2022] PUT /identity/v3/projects/6eb455ba7c994dcaa019094234bc796c/users/af0d167c5ac14b84bac501684ea65a9d/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:54.668844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.679498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d9533-79f7-41e0-a10a-747b1d023413 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:54.755174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb1914-18ea-4c48-a78a-1d763c4a1c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:54.755174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb1914-18ea-4c48-a78a-1d763c4a1c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.755174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb1914-18ea-4c48-a78a-1d763c4a1c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.755174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb1914-18ea-4c48-a78a-1d763c4a1c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.755557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb1914-18ea-4c48-a78a-1d763c4a1c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:54.755557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb1914-18ea-4c48-a78a-1d763c4a1c05 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:54.767177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 892/1785] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:27:54.768412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.782905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0578b6-ff66-4ad0-a94b-33db0b19efdb admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:54.791192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:54.792132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.792537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.792875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] PATH_INFO: `/v3/projects/6eb455ba7c994dcaa019094234bc796c/users/af0d167c5ac14b84bac501684ea65a9d/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.795006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb455ba7c994dcaa019094234bc796c, user_id=af0d167c5ac14b84bac501684ea65a9d, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:54.797721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:54.809944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 894/1786] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:27:54 2022] PUT /identity/v3/projects/6eb455ba7c994dcaa019094234bc796c/users/af0d167c5ac14b84bac501684ea65a9d/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:54.820373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:54.820849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:54.821373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:54.846945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:54.850117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] MFA Rules not processed for user `af0d167c5ac14b84bac501684ea65a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:54.851401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:54.919048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 893/1787] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:54.931684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:54.933333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:54.991393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:54.995357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e67632-bc23-4ee2-be6e-b3dc3e0c402d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:55.006070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:55.006737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:55.007024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:55.007346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:55.017612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:55.019066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:55.029800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 895/1788] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:27:56.483726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:56.493129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c084e8-f4b6-4b79-b208-4fef6d628630 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:56.499522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0d167c5ac14b84bac501684ea65a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6eb455ba7c994dcaa019094234bc796c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:56.500222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:56.500514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:56.500792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:56.501692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:56.502000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:56.503698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:56.516239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 894/1789] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:56.521135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:56.531088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aa7fa-910f-4bab-9cc1-ac5da17f1636 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:56.535730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0d167c5ac14b84bac501684ea65a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6eb455ba7c994dcaa019094234bc796c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:56.536384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:56.536676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:56.536967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:56.537661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:56.539618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:56.640070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 896/1790] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:27:56 2022] POST /identity/v3/projects => generated 354 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:27:56.641315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:56.651480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbec69-398f-4d8a-894d-96831f5b49d3 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:56.708042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0d167c5ac14b84bac501684ea65a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6eb455ba7c994dcaa019094234bc796c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:56.708641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:56.708933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:56.709220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:56.709956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:27:56.710271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:56.711906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:56.723228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 895/1791] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:27:56.764325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:56.774537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8cdbf-9db0-4e70-88c7-4463a0d6e147 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:56.779226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af0d167c5ac14b84bac501684ea65a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6eb455ba7c994dcaa019094234bc796c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:56.780110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:56.780396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:56.780673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] PATH_INFO: `/v3/projects/519d36fc54f34fb98fc3d889d3be6349` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:56.781906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorizing `identity:delete_project(project_id=519d36fc54f34fb98fc3d889d3be6349)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:56.783900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:56.908382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519d36fc54f34fb98fc3d889d3be6349', 'request_id': 'req-d4bec719-738c-41e1-adea-929432ab26b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:56.914071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519d36fc54f34fb98fc3d889d3be6349', 'request_id': 'req-d4bec719-738c-41e1-adea-929432ab26b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:56.992275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519d36fc54f34fb98fc3d889d3be6349', 'request_id': 'req-d4bec719-738c-41e1-adea-929432ab26b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:57.005397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] The token cache is being invalidate because project 519d36fc54f34fb98fc3d889d3be6349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:57.005688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:57.006369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d4bec719-738c-41e1-adea-929432ab26b1 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:57.007956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 897/1792] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:56 2022] DELETE /identity/v3/projects/519d36fc54f34fb98fc3d889d3be6349 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:57.688957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:57.691581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:57.757209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd56de4-78a8-475e-8f27-7284dfde7935 tempest-VolumeQuotaClassesTest-622223246 tempest-VolumeQuotaClassesTest-622223246-project-admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:57.782015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:57.815671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:57.816784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:57.817068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:57.817345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] PATH_INFO: `/v3/users/8449bd44ea5a4c1f899576810e63470f` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:57.818521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8449bd44ea5a4c1f899576810e63470f)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:57.820850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:58.059520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8449bd44ea5a4c1f899576810e63470f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:58.066738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8449bd44ea5a4c1f899576810e63470f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:58.067409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8449bd44ea5a4c1f899576810e63470f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:58.080471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8449bd44ea5a4c1f899576810e63470f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:58.200315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8449bd44ea5a4c1f899576810e63470f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:58.210667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 896/1793] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:57 2022] DELETE /identity/v3/users/8449bd44ea5a4c1f899576810e63470f => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:27:58.215608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:58.216073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:58.216167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:58.233831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:27:58.235038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:58.296463 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 898/1794] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:27:58.306164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:58.307653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:58.366936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5e75d-696d-4112-8360-d123e0714c37 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:58.370606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:58.371263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:58.371581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:58.371862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:58.380902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:58.382341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:58.392390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 897/1795] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:27:58.854358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:58.856023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:27:58.874074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3691f-94f1-4cfc-aa02-2279ba573633 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:58.879608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:58.881102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:58.881611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:58.882090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] PATH_INFO: `/v3/projects/244e62e9b8664e4b95ff54af08a2128e` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:58.886010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244e62e9b8664e4b95ff54af08a2128e)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:58.888781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:59.000484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:59.001716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244e62e9b8664e4b95ff54af08a2128e', 'request_id': 'req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.007762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244e62e9b8664e4b95ff54af08a2128e', 'request_id': 'req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.114334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244e62e9b8664e4b95ff54af08a2128e', 'request_id': 'req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.127322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] The token cache is being invalidate because project 244e62e9b8664e4b95ff54af08a2128e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:27:59.127627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.128456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.130006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 899/1796] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:27:58 2022] DELETE /identity/v3/projects/244e62e9b8664e4b95ff54af08a2128e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:27:59.134063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:27:59.144816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6046c56-68e4-46cd-8b91-71db279750b4 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:27:59.190154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:27:59.201991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:27:59.202892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:27:59.203173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:27:59.203484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] PATH_INFO: `/v3/users/af0d167c5ac14b84bac501684ea65a9d` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:27:59.204662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0d167c5ac14b84bac501684ea65a9d)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:27:59.206575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:27:59.408513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0d167c5ac14b84bac501684ea65a9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.415183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0d167c5ac14b84bac501684ea65a9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.415886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0d167c5ac14b84bac501684ea65a9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.427907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0d167c5ac14b84bac501684ea65a9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.506336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f6104ef-b28e-467d-9047-bc2e23f5251e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0d167c5ac14b84bac501684ea65a9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:27:59.514577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 898/1797] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:27:59 2022] DELETE /identity/v3/users/af0d167c5ac14b84bac501684ea65a9d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:00.068524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:00.070131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:00.083748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5aebe-c1b4-415a-a922-16d7bb3c1393 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:00.141949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:00.142820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:00.143106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:00.143419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] PATH_INFO: `/v3/projects/6eb455ba7c994dcaa019094234bc796c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:00.144607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb455ba7c994dcaa019094234bc796c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:00.146527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:00.259500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb455ba7c994dcaa019094234bc796c', 'request_id': 'req-032f64b0-46ae-48c6-9ea4-b561274e7991'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:00.265929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb455ba7c994dcaa019094234bc796c', 'request_id': 'req-032f64b0-46ae-48c6-9ea4-b561274e7991'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:00.348613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb455ba7c994dcaa019094234bc796c', 'request_id': 'req-032f64b0-46ae-48c6-9ea4-b561274e7991'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:00.364335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] The token cache is being invalidate because project 6eb455ba7c994dcaa019094234bc796c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:00.364675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:00.365287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-032f64b0-46ae-48c6-9ea4-b561274e7991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:00.367077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 900/1798] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:00 2022] DELETE /identity/v3/projects/6eb455ba7c994dcaa019094234bc796c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:00.371977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:00.372435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:00.372544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:00.390972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:00.392188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:00.452962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 899/1799] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:00.460596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:00.460894 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:00.461185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:00.480099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:00.482847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:00.496101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 901/1800] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:00.500263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:00.510179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e1d46-d12f-4026-8c5e-d84776be1ed5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:00.514817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:00.516620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:00.516620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:00.516620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:00.516985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:00.517074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:00.518617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:00.530659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 900/1801] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:00.534934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:00.544380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6294899-aaab-49b2-b807-204c9ffedc86 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:00.549532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:00.550181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:00.550469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:00.550744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:00.551482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:00.553386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:00.669914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 902/1802] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:00 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:00.673850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:00.684231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6316-8a5f-4da5-8981-0af8b1ef1a40 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:00.738900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:00.745720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:00.746447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:00.746675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:00.746954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:00.747705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:00.751645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:01.473674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 901/1803] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:00 2022] POST /identity/v3/users => generated 287 bytes in 800 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:01.485082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:01.496009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2b11-8165-49f7-88e1-49c47a7f5664 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:01.500748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:01.501416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:01.501704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:01.501984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:01.502716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:01.504392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:01.516665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 903/1804] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:01.521012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:01.531772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b6e9-ee2f-49aa-99b1-244a9f0c169f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:01.536269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5d9b9-9b38-47af-b065-7065b7c1b891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:01.536993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5d9b9-9b38-47af-b065-7065b7c1b891 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:01.537276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5d9b9-9b38-47af-b065-7065b7c1b891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:01.537552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5d9b9-9b38-47af-b065-7065b7c1b891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:01.538276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5d9b9-9b38-47af-b065-7065b7c1b891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:01.542492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5d9b9-9b38-47af-b065-7065b7c1b891 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:01.555415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 902/1805] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:01.558971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:01.568884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783a34-bd84-4a78-9990-3ad667a3ffb1 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:01.573550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:01.574446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:01.574714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:01.574995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] PATH_INFO: `/v3/projects/a6235b2a54ef4164aa1e1106e819bc17/users/6b229a040ed74de0a848cdd9970058c8/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:01.590612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6235b2a54ef4164aa1e1106e819bc17, user_id=6b229a040ed74de0a848cdd9970058c8, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:01.593981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:01.892905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 904/1806] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:01 2022] PUT /identity/v3/projects/a6235b2a54ef4164aa1e1106e819bc17/users/6b229a040ed74de0a848cdd9970058c8/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:01.899374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd28a-b0eb-47ee-9666-da320acd4170 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:01.900820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd28a-b0eb-47ee-9666-da320acd4170 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:01.904524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fd28a-b0eb-47ee-9666-da320acd4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:01.942677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-437fd28a-b0eb-47ee-9666-da320acd4170 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:01.944010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-437fd28a-b0eb-47ee-9666-da320acd4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:02.049301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 903/1807] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:02.059869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:02.061453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:02.067533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:02.169653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70e890-e4e2-4d4e-8816-3f5dca01fb90 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:02.175735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:02.178398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:02.178937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:02.179409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:02.192609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:02.195293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:02.210902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 905/1808] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:04.801803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:04.802337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:04.802337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:04.830542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] MFA Rules not processed for user `6b229a040ed74de0a848cdd9970058c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:04.832688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:04.988889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 904/1809] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:04.996059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.020929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006f805-3b2e-4e31-be25-b0622f82e909 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:05.038355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:05.039294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:05.039888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:05.040241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:05.041151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:05.049591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:05.255123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 906/1810] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:04 2022] POST /identity/v3/projects => generated 371 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:05.259227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.271069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536870e9-1f7b-47a3-85fe-1d4a16224cc4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:05.337522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:05.338210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:05.338498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:05.341064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:05.341064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:05.341746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:05.351159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:05.488892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 905/1811] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:05 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:05.493122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.508813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f818c-dbe9-4381-b924-15446a32fff6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:05.528010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:05.528744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:05.529582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:05.529909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:05.530644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:05.533223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:05.566010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 907/1812] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:05.569922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.582886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ad12b-0f57-4ad6-a69a-554a1753cfab admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:05.604880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:05.605921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:05.606152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:05.609486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] PATH_INFO: `/v3/projects/9aceace1f9a7496da5aff1c39a1a826c/users/f450100f92cc4fc68388ae89224cd467/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:05.643355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aceace1f9a7496da5aff1c39a1a826c, user_id=f450100f92cc4fc68388ae89224cd467, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:05.647568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:05.742891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 906/1813] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:05 2022] PUT /identity/v3/projects/9aceace1f9a7496da5aff1c39a1a826c/users/f450100f92cc4fc68388ae89224cd467/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:05.745230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.762620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32227acd-0715-4e77-a717-8a9f560b3b1d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:05.831624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:05.832299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:05.832582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:05.832860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:05.833583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:05.835347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:05.851225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 908/1814] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:05.852309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.866977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2c78-135d-49aa-bb59-8a331320f0b4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:05.872166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fe428-9b7f-49b0-af30-cddef48a0ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:05.873124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fe428-9b7f-49b0-af30-cddef48a0ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:05.873415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fe428-9b7f-49b0-af30-cddef48a0ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:05.873704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fe428-9b7f-49b0-af30-cddef48a0ee3 admin admin] PATH_INFO: `/v3/projects/9aceace1f9a7496da5aff1c39a1a826c/users/f450100f92cc4fc68388ae89224cd467/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:05.875744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fe428-9b7f-49b0-af30-cddef48a0ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aceace1f9a7496da5aff1c39a1a826c, user_id=f450100f92cc4fc68388ae89224cd467, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:05.878418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fe428-9b7f-49b0-af30-cddef48a0ee3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:05.889814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 907/1815] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:05 2022] PUT /identity/v3/projects/9aceace1f9a7496da5aff1c39a1a826c/users/f450100f92cc4fc68388ae89224cd467/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:05.946224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:05.948150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:05.961481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47d68c-0755-40f0-b840-9ac611284328 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:06.023041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:06.024015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:06.024308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:06.024611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] PATH_INFO: `/v3/users/cbd9c02d0eb3462da854b75745861101` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:06.026103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd9c02d0eb3462da854b75745861101)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:06.028092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:06.041052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:06.314693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd9c02d0eb3462da854b75745861101'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:06.315496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd9c02d0eb3462da854b75745861101'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:06.327708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd9c02d0eb3462da854b75745861101'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:06.334755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd9c02d0eb3462da854b75745861101'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:06.432305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd9c02d0eb3462da854b75745861101'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:06.439267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 909/1816] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:05 2022] DELETE /identity/v3/users/cbd9c02d0eb3462da854b75745861101 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:06.445566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:06.445866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:06.446235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:06.464608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:06.465840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:06.525860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 908/1817] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:06.536086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:06.537624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:06.596796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216619ac-e4df-4499-b18c-5eb722c9a3d8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:06.613646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2257f9-d059-4e60-9ea5-e78ca355ef7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:06.614347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2257f9-d059-4e60-9ea5-e78ca355ef7e service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:06.614603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2257f9-d059-4e60-9ea5-e78ca355ef7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:06.614879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2257f9-d059-4e60-9ea5-e78ca355ef7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:06.628586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2257f9-d059-4e60-9ea5-e78ca355ef7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:06.631537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2257f9-d059-4e60-9ea5-e78ca355ef7e service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:06.641445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 910/1818] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:07.182227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:07.183718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:07.195894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc85d48-11af-4721-ad51-51a847b42089 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:07.200535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:07.201544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:07.201832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:07.202192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] PATH_INFO: `/v3/projects/9407fbe87482460a85db70f33ce89b31` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:07.204723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9407fbe87482460a85db70f33ce89b31)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:07.206650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:07.343972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9407fbe87482460a85db70f33ce89b31', 'request_id': 'req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.438236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9407fbe87482460a85db70f33ce89b31', 'request_id': 'req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.447114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9407fbe87482460a85db70f33ce89b31', 'request_id': 'req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.466436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] The token cache is being invalidate because project 9407fbe87482460a85db70f33ce89b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:07.468026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.469416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.476701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 909/1819] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:07 2022] DELETE /identity/v3/projects/9407fbe87482460a85db70f33ce89b31 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:07.486735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:07.486735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:07.486735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:07.487900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:07.497143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:07.522674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] MFA Rules not processed for user `f450100f92cc4fc68388ae89224cd467`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:07.524558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:07.568855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:07.601458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e104a-9ed9-4c44-9d15-a7fc8daac734 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:07.616002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:07.616954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:07.618481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:07.618481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] PATH_INFO: `/v3/users/bf786646f4694be0963fe4f6077d0f7c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:07.619198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf786646f4694be0963fe4f6077d0f7c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:07.621170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:07.624493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 911/1820] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:07.637952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:07.640747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:07.667639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:07.716063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a43560-f083-42c9-acae-a0d23b9c18d4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:07.720427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:07.721104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:07.721384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:07.721659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:07.723495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:07.784596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:07.786033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:07.794959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 912/1821] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:07.887735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf786646f4694be0963fe4f6077d0f7c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.898016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf786646f4694be0963fe4f6077d0f7c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.898770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf786646f4694be0963fe4f6077d0f7c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:07.917449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf786646f4694be0963fe4f6077d0f7c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.009646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0cbe2af4-3652-4e99-9e96-632ec9acbaf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf786646f4694be0963fe4f6077d0f7c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.017508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 910/1822] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:07 2022] DELETE /identity/v3/users/bf786646f4694be0963fe4f6077d0f7c => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:08.569159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:08.581754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c14abf-118a-435a-80b1-7ec85ad78663 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:08.660302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:08.661222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:08.661439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:08.661834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] PATH_INFO: `/v3/projects/91c16b8d08294f3c958f3657bbc4e6a3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:08.663032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c16b8d08294f3c958f3657bbc4e6a3)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:08.664995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:08.817636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c16b8d08294f3c958f3657bbc4e6a3', 'request_id': 'req-a1c1ac46-edc3-4772-a26f-012da561bf7c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.824398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c16b8d08294f3c958f3657bbc4e6a3', 'request_id': 'req-a1c1ac46-edc3-4772-a26f-012da561bf7c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.913945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c16b8d08294f3c958f3657bbc4e6a3', 'request_id': 'req-a1c1ac46-edc3-4772-a26f-012da561bf7c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.926059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] The token cache is being invalidate because project 91c16b8d08294f3c958f3657bbc4e6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:08.926374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.927000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a1c1ac46-edc3-4772-a26f-012da561bf7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:08.928536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 913/1823] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:08 2022] DELETE /identity/v3/projects/91c16b8d08294f3c958f3657bbc4e6a3 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:08.933468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:08.933943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:08.934046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:08.953087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:08.954301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:08.988788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:09.014431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 911/1824] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:09.022541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.022834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.023117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.041292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:09.042516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:09.051018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 914/1825] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:09.055395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.065953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788bb3d-a434-45e9-81db-64b1b85ec2b9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.070655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.071262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.071573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.071853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.072712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:09.073026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.074622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.086359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 912/1826] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:09.090792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.099956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752a410-93b0-40a0-a8a9-ca61b1e554bb None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.104721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.105371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.105653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.105928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.106630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.109404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.306460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 915/1827] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:09 2022] POST /identity/v3/projects => generated 401 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:09.308102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.318795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07161d29-929a-415a-a3d1-b8a128606162 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.376214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:09.388702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.389383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.389667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.389951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.390986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.392958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.507165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 913/1828] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:09 2022] POST /identity/v3/users => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:09.511554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.521645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c22b4-16e3-4ec0-ae18-4fe983f2c48a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.527005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.527705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.527989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.528266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.529533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.531151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.541645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 916/1829] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:09.548329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.558322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948015-5f82-4491-ba1b-5c7dd63dddbc admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.563108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d6a1-eaf3-41c5-a49c-da84daf68b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.563815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d6a1-eaf3-41c5-a49c-da84daf68b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.564343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d6a1-eaf3-41c5-a49c-da84daf68b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.565037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d6a1-eaf3-41c5-a49c-da84daf68b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.565849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d6a1-eaf3-41c5-a49c-da84daf68b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.567518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d6a1-eaf3-41c5-a49c-da84daf68b52 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.578260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 914/1830] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:09.582415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.593131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9ddd-3f64-4e43-96ad-c7a4963c736f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.597636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.598510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.598793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.602619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] PATH_INFO: `/v3/projects/d4e8f9a6ce044d40b5473da01a986ae8/users/fb25f91570824edca8680e36f11dc8db/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.615368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e8f9a6ce044d40b5473da01a986ae8, user_id=fb25f91570824edca8680e36f11dc8db, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.618024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.714618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 917/1831] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:09 2022] PUT /identity/v3/projects/d4e8f9a6ce044d40b5473da01a986ae8/users/fb25f91570824edca8680e36f11dc8db/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:09.726696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.727056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.727424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.753496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] MFA Rules not processed for user `fb25f91570824edca8680e36f11dc8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:09.758328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:09.820325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 915/1832] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:09.825193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.888132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb88ea5-bc8b-4497-918a-eb7330db564f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:09.893384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:09.894041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:09.896785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:09.896785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:09.896785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:09.897230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:09.979821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 918/1833] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:09 2022] POST /identity/v3/projects => generated 401 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:09.984222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:09.995996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb5dc1-e1e0-468f-b082-93f881d332e9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.063813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.064497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.064787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.065070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.065780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:10.067755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:10.183277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 916/1834] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:09 2022] POST /identity/v3/users => generated 307 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:10.187579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:10.200509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d63638-4585-4856-bcc6-1d36bdd2c41d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.205506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.206172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.206464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.206742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.207510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:10.209267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:10.224407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 919/1835] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:10.225649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:10.237511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c1861-a410-4e81-8c37-e8447a3eeb87 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.242177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.243086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.243408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.243696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] PATH_INFO: `/v3/projects/24e6587b42174a699154061848d2fa08/users/27ff9dae5db5490a8b9c121694dcfb7f/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.261396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e6587b42174a699154061848d2fa08, user_id=27ff9dae5db5490a8b9c121694dcfb7f, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:10.264109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:10.400412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 917/1836] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:10 2022] PUT /identity/v3/projects/24e6587b42174a699154061848d2fa08/users/27ff9dae5db5490a8b9c121694dcfb7f/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:10.404360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:10.417250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520ce37-8d67-413b-b1b5-0a5d583457ad admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.493311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259c77a-c30a-4fb4-a01f-c58274dd4193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.494159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259c77a-c30a-4fb4-a01f-c58274dd4193 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.494514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259c77a-c30a-4fb4-a01f-c58274dd4193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.494860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259c77a-c30a-4fb4-a01f-c58274dd4193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.495704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259c77a-c30a-4fb4-a01f-c58274dd4193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:10.497446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259c77a-c30a-4fb4-a01f-c58274dd4193 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:10.511825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 920/1837] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:10.512774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:10.523494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a2c02-f54b-4905-bf94-b8b8eaf55c9c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.528391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.529108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.529397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.529673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] PATH_INFO: `/v3/projects/24e6587b42174a699154061848d2fa08/users/27ff9dae5db5490a8b9c121694dcfb7f/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.532414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e6587b42174a699154061848d2fa08, user_id=27ff9dae5db5490a8b9c121694dcfb7f, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:10.535132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:10.546027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 918/1838] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:10 2022] PUT /identity/v3/projects/24e6587b42174a699154061848d2fa08/users/27ff9dae5db5490a8b9c121694dcfb7f/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:10.556259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.556555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.556838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.569125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:10.584194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] MFA Rules not processed for user `27ff9dae5db5490a8b9c121694dcfb7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:10.585667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:10.657728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 921/1839] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:10.668399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:10.670090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:10.679981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:10.730303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e510618-4fc9-4266-bd14-43a834040c0f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.734259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a2339-8e2d-464c-b4b9-0bc82f4c813f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.734907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2339-8e2d-464c-b4b9-0bc82f4c813f service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.735194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2339-8e2d-464c-b4b9-0bc82f4c813f service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.735505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a2339-8e2d-464c-b4b9-0bc82f4c813f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:10.744338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a2339-8e2d-464c-b4b9-0bc82f4c813f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:10.745779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a2339-8e2d-464c-b4b9-0bc82f4c813f service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:10.756011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 919/1840] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:10.979432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:10.989124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f60f-2d66-4741-80fc-761adcdbefcc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:10.992852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065010fc-3b00-43fb-a66c-bd8a7779df0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:10.995931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065010fc-3b00-43fb-a66c-bd8a7779df0d service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:10.995931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065010fc-3b00-43fb-a66c-bd8a7779df0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:10.995931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065010fc-3b00-43fb-a66c-bd8a7779df0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:11.051546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065010fc-3b00-43fb-a66c-bd8a7779df0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:11.053337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065010fc-3b00-43fb-a66c-bd8a7779df0d service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:11.063908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 922/1841] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:13.286325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deb4a1-548a-434c-b612-460844c52cc2 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:13.286857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deb4a1-548a-434c-b612-460844c52cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:13.287875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deb4a1-548a-434c-b612-460844c52cc2 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:13.290316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 920/1842] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:28:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:28:13.302161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f47e2-1431-4626-99b3-218b9324c57b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:13.302697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f47e2-1431-4626-99b3-218b9324c57b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:13.303488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f47e2-1431-4626-99b3-218b9324c57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:13.324676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3e8f47e2-1431-4626-99b3-218b9324c57b None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:13.326021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3e8f47e2-1431-4626-99b3-218b9324c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:13.501211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 923/1843] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:28:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:13.606096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4deb4a1-548a-434c-b612-460844c52cc2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:13.609785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b4deb4a1-548a-434c-b612-460844c52cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:13.670260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4deb4a1-548a-434c-b612-460844c52cc2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:13.691157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aed6b3-ac56-408a-945d-db647c0a2731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:13.691862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aed6b3-ac56-408a-945d-db647c0a2731 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:13.692146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aed6b3-ac56-408a-945d-db647c0a2731 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:13.697957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aed6b3-ac56-408a-945d-db647c0a2731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:13.715027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aed6b3-ac56-408a-945d-db647c0a2731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:13.716965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aed6b3-ac56-408a-945d-db647c0a2731 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:13.766260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 921/1844] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:13.802309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f878549-66d6-4c0d-b324-551422b533b6 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:13.802606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f878549-66d6-4c0d-b324-551422b533b6 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:13.802898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f878549-66d6-4c0d-b324-551422b533b6 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:13.804605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 924/1845] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:28:13.840447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:13.841092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:13.841516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:13.882535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:13.884533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:13.919433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 922/1846] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:28:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:13.969656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f878549-66d6-4c0d-b324-551422b533b6 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:13.971960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2f878549-66d6-4c0d-b324-551422b533b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:14.011118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f878549-66d6-4c0d-b324-551422b533b6 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:14.026583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:14.028624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:14.028912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:14.029199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:14.054787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:14.059169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:14.113197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 925/1847] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:14.165228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:14.182561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164f804-2a88-4866-b9b0-c4563bd0cbd9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:14.200093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:14.200946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:14.201372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:14.201826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:14.207357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:14.350520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:14.353744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15c0e8-e399-4d01-90b3-fbdd609024e1 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:14.365851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 923/1848] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:21.973714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:21.983830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:22.047214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304df0f1-5575-4d3e-9962-f79e279102f1 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:22.052149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:22.053191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:22.053472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:22.054603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] PATH_INFO: `/v3/users/fb25f91570824edca8680e36f11dc8db` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:22.054931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb25f91570824edca8680e36f11dc8db)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:22.056904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:22.259543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb25f91570824edca8680e36f11dc8db'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:22.260201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb25f91570824edca8680e36f11dc8db'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:22.272999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb25f91570824edca8680e36f11dc8db'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:22.279991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb25f91570824edca8680e36f11dc8db'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:22.346338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb25f91570824edca8680e36f11dc8db'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:22.353160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 926/1849] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:21 2022] DELETE /identity/v3/users/fb25f91570824edca8680e36f11dc8db => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:22.358026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:22.358552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:22.358654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:22.378989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:22.380401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:22.449711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 924/1850] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:22.459166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:22.460621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:22.521765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a8085-5b8c-47c2-9415-117d71691f54 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:22.525594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:22.526221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:22.526526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:22.526808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:22.536870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:22.538305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:22.552296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 927/1851] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:22.986714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:22.988957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:23.003972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc5137-3b83-4ddc-bcbb-ff57c31e7fe8 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:23.011348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:23.012233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:23.012516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:23.012793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] PATH_INFO: `/v3/projects/d4e8f9a6ce044d40b5473da01a986ae8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:23.014244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e8f9a6ce044d40b5473da01a986ae8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:23.016790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:23.043207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:23.173607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e8f9a6ce044d40b5473da01a986ae8', 'request_id': 'req-fdc40ecc-f755-4ca4-a00a-156efc04613e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.262827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e8f9a6ce044d40b5473da01a986ae8', 'request_id': 'req-fdc40ecc-f755-4ca4-a00a-156efc04613e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.273881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e8f9a6ce044d40b5473da01a986ae8', 'request_id': 'req-fdc40ecc-f755-4ca4-a00a-156efc04613e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.286947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] The token cache is being invalidate because project d4e8f9a6ce044d40b5473da01a986ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:23.287423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.288386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.290155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 925/1852] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:22 2022] DELETE /identity/v3/projects/d4e8f9a6ce044d40b5473da01a986ae8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:23.295163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:23.306625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce081e-ba7c-455f-bd61-3fa7f7671eaf service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:23.341119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:23.371533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:23.372401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:23.372680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:23.372951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] PATH_INFO: `/v3/users/27ff9dae5db5490a8b9c121694dcfb7f` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:23.374133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ff9dae5db5490a8b9c121694dcfb7f)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:23.376150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:23.568850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ff9dae5db5490a8b9c121694dcfb7f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.569522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ff9dae5db5490a8b9c121694dcfb7f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.588081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ff9dae5db5490a8b9c121694dcfb7f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.595756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ff9dae5db5490a8b9c121694dcfb7f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.714459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b08465fb-6727-43f6-a20b-a1d350120e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ff9dae5db5490a8b9c121694dcfb7f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:23.721245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 928/1853] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:23 2022] DELETE /identity/v3/users/27ff9dae5db5490a8b9c121694dcfb7f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:24.138294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.139848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:24.152888 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40ecc-f755-4ca4-a00a-156efc04613e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.217765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.219153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.219478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.219746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] PATH_INFO: `/v3/projects/24e6587b42174a699154061848d2fa08` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.221006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e6587b42174a699154061848d2fa08)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.222942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:24.322500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e6587b42174a699154061848d2fa08', 'request_id': 'req-6f3767e2-ef29-4c19-a6aa-6024a7595a67'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:24.415147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e6587b42174a699154061848d2fa08', 'request_id': 'req-6f3767e2-ef29-4c19-a6aa-6024a7595a67'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:24.421345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e6587b42174a699154061848d2fa08', 'request_id': 'req-6f3767e2-ef29-4c19-a6aa-6024a7595a67'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:24.433826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] The token cache is being invalidate because project 24e6587b42174a699154061848d2fa08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:24.434122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:24.436072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f3767e2-ef29-4c19-a6aa-6024a7595a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:24.436424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 926/1854] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:24 2022] DELETE /identity/v3/projects/24e6587b42174a699154061848d2fa08 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:24.441711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.442184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.442271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.460690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:24.461906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:24.524365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 929/1855] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:24.532632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.532922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.533203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.551930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:24.553115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:24.560907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 927/1856] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:24.565152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.574783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79b712-181d-4009-a391-71ecc213a59b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.580168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.580822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.581108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.592919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.592919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:24.592919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.592919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:24.596191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 930/1857] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:24.601053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.610745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102dc84-8955-4ba8-bf81-a6711b3b9fe5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.615471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.616544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.616834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.617112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.617805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.619783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:24.686176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 928/1858] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:24 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:24.690207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.700199 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c0bce-3aa8-43bc-b9dd-cc0406455ba6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.737878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:24.760662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.761329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.761610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.761886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.762593 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.764626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:24.860483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 931/1859] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:24 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:24.861669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.872052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3abcaf-f8e1-4e2f-b396-899b39be8e70 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.876710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.877372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.877659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.878015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.878842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.880476 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:24.882710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:24.892369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 929/1860] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:24.896464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.908666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd9248-c1ac-4d6e-bf96-65551e331551 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.913412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57b284-de74-4a63-bb19-fd67704de444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.914076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57b284-de74-4a63-bb19-fd67704de444 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.914367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57b284-de74-4a63-bb19-fd67704de444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.914643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57b284-de74-4a63-bb19-fd67704de444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.915394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57b284-de74-4a63-bb19-fd67704de444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.917006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57b284-de74-4a63-bb19-fd67704de444 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:24.930857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 932/1861] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:24.932437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:24.943250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fabf8d-9f26-43e8-9d55-8e9c5823b962 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:24.947937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:24.948819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:24.949104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:24.949383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] PATH_INFO: `/v3/projects/45eaca6c57b049f6b11c604b2bcbb211/users/fa19d147eb1a4bc69097372d2737f9a0/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:24.964242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45eaca6c57b049f6b11c604b2bcbb211, user_id=fa19d147eb1a4bc69097372d2737f9a0, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:24.966902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.068959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 930/1862] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:24 2022] PUT /identity/v3/projects/45eaca6c57b049f6b11c604b2bcbb211/users/fa19d147eb1a4bc69097372d2737f9a0/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:25.080589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.080885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.081219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.108482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] MFA Rules not processed for user `fa19d147eb1a4bc69097372d2737f9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:25.110430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:25.175431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 933/1863] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:25.179959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.240539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b6d3a-a220-40b4-a019-cd4771f16177 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:25.245399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:25.246056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.246343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.246623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.247323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:25.249240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.350006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 931/1864] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:25 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:25.353862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.365475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73674066-52fc-45bc-b2a7-ee87b1cbc82b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:25.430323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:25.431006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.431293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.431601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.432316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:25.434793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.512373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 934/1865] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:25 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:25.513975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.524600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4baea-e82c-42bd-b4ef-455f275cd2d3 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:25.529554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:25.530347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.530701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.531117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.531887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:25.533492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.544841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 932/1866] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:25.549516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.559684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5637d66-b8ee-4764-a07f-3862552f1c88 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:25.565062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:25.565978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.566263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.566537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] PATH_INFO: `/v3/projects/49152295ab4446769929ce8408e40d17/users/1f01d8f2e7d0479d8aa6660c178988b8/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.581329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49152295ab4446769929ce8408e40d17, user_id=1f01d8f2e7d0479d8aa6660c178988b8, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:25.584025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.685909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 935/1867] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:25 2022] PUT /identity/v3/projects/49152295ab4446769929ce8408e40d17/users/1f01d8f2e7d0479d8aa6660c178988b8/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:25.690441 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.700306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334b66f-a878-4cb9-b7fa-9799dbde61fe admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:25.759220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bb8c8-311f-4f48-b7f8-87cbc71ce0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:25.759925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bb8c8-311f-4f48-b7f8-87cbc71ce0db admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.760208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bb8c8-311f-4f48-b7f8-87cbc71ce0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.760487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bb8c8-311f-4f48-b7f8-87cbc71ce0db admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.761222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bb8c8-311f-4f48-b7f8-87cbc71ce0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:25.763005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bb8c8-311f-4f48-b7f8-87cbc71ce0db admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.773597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 933/1868] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:25.777602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.788356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e301fd9-44e7-43a5-a883-f5aafcae4d2a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:25.792981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:25.793885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.794166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.794441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] PATH_INFO: `/v3/projects/49152295ab4446769929ce8408e40d17/users/1f01d8f2e7d0479d8aa6660c178988b8/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.796421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49152295ab4446769929ce8408e40d17, user_id=1f01d8f2e7d0479d8aa6660c178988b8, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:25.799744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:25.805277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:25.810448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 936/1869] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:25 2022] PUT /identity/v3/projects/49152295ab4446769929ce8408e40d17/users/1f01d8f2e7d0479d8aa6660c178988b8/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:25.827228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:25.827552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:25.827861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:25.854533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] MFA Rules not processed for user `1f01d8f2e7d0479d8aa6660c178988b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:25.855817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:25.890030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:25.926099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 934/1870] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:25.937444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:25.939142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:26.000759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66c5a8-2b22-4afd-8294-fd087ef2fec7 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:26.004471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:26.005111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:26.005394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:26.005668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:26.020775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:26.022352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:26.032731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 937/1871] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:26.258866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:26.268898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86b33-1499-4073-ad34-0dbfa9972b0a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:26.272489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:26.273345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:26.273635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:26.273917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:26.329100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:26.330533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:26.339500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 935/1872] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:29.373064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:29.374922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:29.442424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9626b-d733-4638-9cae-930c24c5de31 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:29.447147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:29.448059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:29.448336 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:29.448611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] PATH_INFO: `/v3/users/2af32f291664436ea90593be86310003` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:29.449975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af32f291664436ea90593be86310003)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:29.451938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:29.608601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:29.610645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:29.624572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26afc9b-b3ae-4b09-89cc-4fc7469ae7c4 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:29.629864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:29.630768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:29.631068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:29.631363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] PATH_INFO: `/v3/users/b51a244ac5de4191b282af104758b6b8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:29.632581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51a244ac5de4191b282af104758b6b8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:29.634527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:29.680828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af32f291664436ea90593be86310003'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.681515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af32f291664436ea90593be86310003'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.695006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af32f291664436ea90593be86310003'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.701818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af32f291664436ea90593be86310003'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.781082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-211b7a6d-f58c-4a44-83e4-788636346334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af32f291664436ea90593be86310003'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.788343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 938/1873] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:29 2022] DELETE /identity/v3/users/2af32f291664436ea90593be86310003 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:29.792550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:29.792897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:29.793181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:29.810349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:29.811586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:29.824716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:29.871807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 939/1874] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:29.882595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:29.884384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:29.889542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51a244ac5de4191b282af104758b6b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.916415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51a244ac5de4191b282af104758b6b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.917375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51a244ac5de4191b282af104758b6b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.934119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51a244ac5de4191b282af104758b6b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:29.948903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4226d59-4c46-4616-8bac-05ea968c491e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:29.952726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:29.953377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:29.954359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:29.954950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:29.957165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:29.969005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:29.971018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a12386-46be-4e92-8f94-6dadde23751c service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:29.980439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 940/1875] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:30.049210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51a244ac5de4191b282af104758b6b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:30.057098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 936/1876] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:29 2022] DELETE /identity/v3/users/b51a244ac5de4191b282af104758b6b8 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:30.062145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:30.063558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:30.064080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:30.093422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:30.095516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:30.209204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 941/1877] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:30.342849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:30.344975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:30.434250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:30.472030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4207-cb14-4a08-a816-48e78fc9601d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:30.480026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:30.482020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:30.482371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:30.483993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:30.498959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:30.500933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:30.513439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 937/1878] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:30.699878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:30.704354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:30.725803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af984c6-6809-40cb-8ebf-272e1d81ac09 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:30.746361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:30.747812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:30.748115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:30.748476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] PATH_INFO: `/v3/projects/ff19af64e879451ea621d313631390c4` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:30.749616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff19af64e879451ea621d313631390c4)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:30.751735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:30.925347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff19af64e879451ea621d313631390c4', 'request_id': 'req-61c91110-b915-4b22-8546-4fec5528443e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:30.941273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff19af64e879451ea621d313631390c4', 'request_id': 'req-61c91110-b915-4b22-8546-4fec5528443e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:30.949144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:31.039635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff19af64e879451ea621d313631390c4', 'request_id': 'req-61c91110-b915-4b22-8546-4fec5528443e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.060455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] The token cache is being invalidate because project ff19af64e879451ea621d313631390c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:31.061146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.064421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.078089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 942/1879] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:30 2022] DELETE /identity/v3/projects/ff19af64e879451ea621d313631390c4 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:31.082836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:31.093662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb253e31-6d41-4a3d-a267-e8c6dd8826db service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:31.120263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:31.121940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:31.162590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:31.164397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:31.164881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:31.165271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] PATH_INFO: `/v3/users/c39670d3940b4a21beb9f22bd5936dc7` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:31.167055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39670d3940b4a21beb9f22bd5936dc7)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:31.170719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:31.289707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c91110-b915-4b22-8546-4fec5528443e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:31.298382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:31.300164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:31.300457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:31.300740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] PATH_INFO: `/v3/projects/026a8415ad6e473aadbeb43ce073f9a0` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:31.302151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] RBAC: Authorizing `identity:delete_project(project_id=026a8415ad6e473aadbeb43ce073f9a0)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:31.304953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:31.493603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39670d3940b4a21beb9f22bd5936dc7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.517039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39670d3940b4a21beb9f22bd5936dc7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.518141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39670d3940b4a21beb9f22bd5936dc7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.542654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026a8415ad6e473aadbeb43ce073f9a0', 'request_id': 'req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.556608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39670d3940b4a21beb9f22bd5936dc7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.558006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026a8415ad6e473aadbeb43ce073f9a0', 'request_id': 'req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.657889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7f730d22-7314-4eac-a181-d111d8d012cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39670d3940b4a21beb9f22bd5936dc7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.682396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 938/1880] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:31 2022] DELETE /identity/v3/users/c39670d3940b4a21beb9f22bd5936dc7 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:31.751237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026a8415ad6e473aadbeb43ce073f9a0', 'request_id': 'req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.770562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] The token cache is being invalidate because project 026a8415ad6e473aadbeb43ce073f9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:31.771367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.772377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ebe32e7c-24e0-432b-ba8f-0c821cc6dede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:31.774932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 943/1881] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:31 2022] DELETE /identity/v3/projects/026a8415ad6e473aadbeb43ce073f9a0 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:31.779902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:31.780498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:31.781052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:31.802193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:31.803879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:31.892758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 939/1882] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:31.902437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:31.903467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:31.904058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:31.932517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:31.934455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:31.946041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 944/1883] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:31.952631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:31.963080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:31.980443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629de16-44b8-4ea3-8a24-98cfff89359c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.003953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.005355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.005955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.006562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.008087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:32.008609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.020827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.029247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 940/1884] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:32.033221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.058673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761c0d-7ee1-427f-904c-af8197b3c47a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.079598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.079598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.079598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.079598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.080385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.083553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.256680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.259796 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:32.265136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 945/1885] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:32 2022] POST /identity/v3/projects => generated 365 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:32.270191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.284207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d79c97-72d6-487e-bb93-1be485ed8aa8 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.346380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:32.359604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de40ab-b007-4cce-9c1a-7325641b63c4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.361180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.362063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.362351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.362629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] PATH_INFO: `/v3/projects/d57f667939ed4342a27068dee822ed7f` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.363906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57f667939ed4342a27068dee822ed7f)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.365471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.365809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.366257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.366570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.366852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.367571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.369883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.505958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 946/1886] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:32 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:32.507000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.514537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57f667939ed4342a27068dee822ed7f', 'request_id': 'req-a56ad8d3-8925-4a33-a675-973cb861b17f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:32.516876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2804c-f37a-4cf1-a0e9-d93e780fffc2 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.575112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.576159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.576466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.576750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.577599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.580099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.593996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 947/1887] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:32.595744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.607282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc8bb1-84c2-4d4d-8ca9-b05798094756 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.608575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57f667939ed4342a27068dee822ed7f', 'request_id': 'req-a56ad8d3-8925-4a33-a675-973cb861b17f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:32.617573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.617573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.617573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.617573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.617942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.618012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57f667939ed4342a27068dee822ed7f', 'request_id': 'req-a56ad8d3-8925-4a33-a675-973cb861b17f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:32.618095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.633896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 948/1888] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:32.634237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] The token cache is being invalidate because project d57f667939ed4342a27068dee822ed7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:32.634237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:32.634237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a56ad8d3-8925-4a33-a675-973cb861b17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:32.635233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 941/1889] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:32 2022] DELETE /identity/v3/projects/d57f667939ed4342a27068dee822ed7f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:32.635859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.637819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:32.640084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c8286-b283-4f7f-96ba-670e3c4b3dad None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.640454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c8286-b283-4f7f-96ba-670e3c4b3dad None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.640812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c8286-b283-4f7f-96ba-670e3c4b3dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.652750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810053cb-2f87-46b2-8d12-e09e195dc698 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.659608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029adb-6c36-4757-abd1-11a726006911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.660510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029adb-6c36-4757-abd1-11a726006911 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.660815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029adb-6c36-4757-abd1-11a726006911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.661074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029adb-6c36-4757-abd1-11a726006911 admin admin] PATH_INFO: `/v3/projects/da3d5c0d75054fa383f9033bfb135e6d/users/3c85aa9b1b1c49f7b848cec7b43bd6fc/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.663069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0a3c8286-b283-4f7f-96ba-670e3c4b3dad None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:32.664482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0a3c8286-b283-4f7f-96ba-670e3c4b3dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:32.680554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 942/1890] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:32.689011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029adb-6c36-4757-abd1-11a726006911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3d5c0d75054fa383f9033bfb135e6d, user_id=3c85aa9b1b1c49f7b848cec7b43bd6fc, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.693150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029adb-6c36-4757-abd1-11a726006911 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.693260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22613735-b4f2-4340-9693-271650696a0e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.693463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22613735-b4f2-4340-9693-271650696a0e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.693746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22613735-b4f2-4340-9693-271650696a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.723069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-22613735-b4f2-4340-9693-271650696a0e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:32.726465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-22613735-b4f2-4340-9693-271650696a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:32.738197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 943/1891] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:32.744455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22613735-b4f2-4340-9693-271650696a0e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.759477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22613735-b4f2-4340-9693-271650696a0e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.765415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.766802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.767161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.767545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.768353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:32.768727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.770545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:32.791813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 944/1892] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:32.793616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:32.806865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916c3f4-0e66-4376-a2d5-abd12927ba01 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:32.815059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 949/1893] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:32 2022] PUT /identity/v3/projects/da3d5c0d75054fa383f9033bfb135e6d/users/3c85aa9b1b1c49f7b848cec7b43bd6fc/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:32.825748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.826291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.826581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.900339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] MFA Rules not processed for user `3c85aa9b1b1c49f7b848cec7b43bd6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:32.901554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:32.920946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:32.924056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:32.924056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:32.924056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:32.924056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:32.925063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.013422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 950/1894] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:33.014614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.026325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3a500-ac0a-4d5d-82e6-d4a9f816b193 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.031246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.032951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.032951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.033238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.033744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.035740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.058160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 945/1895] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:32 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:33.064694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.127690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcce25b-be8f-471a-92f3-68a1d3cfc1d9 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.132827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.133506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.133793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.134071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.134771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.137368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.146672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 951/1896] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:33.150198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.214342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bdfe0-70d6-4bf2-8777-b6f8bf39c828 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.219011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.219739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.220023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.220303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.221029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.223019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.237966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 946/1897] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:33.239087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.248889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427ba05-fdd6-44c3-97cb-93bbdde4db99 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.254107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.254759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.255178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.255820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.256549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.258145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.268834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 947/1898] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:33.272891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.282411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8aab9-5acc-4724-ae64-db458e126d35 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.287024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.287946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.288232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.288504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.289199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.290826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.301590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 948/1899] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:33.305531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.315614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0efb18-1ba6-4294-95d1-3bc697bab0b7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.320194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3383-1319-467b-84f3-a4ac2b630737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.321036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3383-1319-467b-84f3-a4ac2b630737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.324656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3383-1319-467b-84f3-a4ac2b630737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.324656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3383-1319-467b-84f3-a4ac2b630737 admin admin] PATH_INFO: `/v3/projects/09966bf372384c24a7f8087c322dbc8f/users/35c9271ba32e4197b96ab6ed3eeb9d00/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.336298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3383-1319-467b-84f3-a4ac2b630737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09966bf372384c24a7f8087c322dbc8f, user_id=35c9271ba32e4197b96ab6ed3eeb9d00, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.338903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3383-1319-467b-84f3-a4ac2b630737 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.359378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 952/1900] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:33.363123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.370734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca8f9d-1e1b-4658-9e78-d71f5b8a4bf6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.376188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.376853 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.377143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.377498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.378263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.379889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.395716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 953/1901] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:33.399820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.411382 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67924661-8dcb-444c-861c-6cd92ed200bf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.419458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.419458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.419458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.419458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] PATH_INFO: `/v3/projects/97def3b1b7c7427c873910ef0275b81e/users/9ce64d49987c409093b3958c9cd9181c/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.433628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97def3b1b7c7427c873910ef0275b81e, user_id=9ce64d49987c409093b3958c9cd9181c, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.436440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.445044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 949/1902] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:33 2022] PUT /identity/v3/projects/09966bf372384c24a7f8087c322dbc8f/users/35c9271ba32e4197b96ab6ed3eeb9d00/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:33.456307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.456604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.456902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.482733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] MFA Rules not processed for user `35c9271ba32e4197b96ab6ed3eeb9d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:33.484020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:33.536920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 954/1903] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:33 2022] PUT /identity/v3/projects/97def3b1b7c7427c873910ef0275b81e/users/9ce64d49987c409093b3958c9cd9181c/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:33.540489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.559755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 950/1904] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:33.566440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.608426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967417a-97ac-499a-82df-f64f0aec2778 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.613750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.614421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.614741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.615024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.615780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.617407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.631498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f54964-adb9-4c30-aad4-3049a31869a0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.634718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 955/1905] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:33.635860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.638878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.639592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.639877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.640154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.640844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.642789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.645337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f1d1b-6259-440d-99b4-9f7384050135 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.650130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aabdf-3f90-4482-8b75-51a384fea7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.651011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aabdf-3f90-4482-8b75-51a384fea7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.651329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aabdf-3f90-4482-8b75-51a384fea7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.651615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aabdf-3f90-4482-8b75-51a384fea7ab admin admin] PATH_INFO: `/v3/projects/97def3b1b7c7427c873910ef0275b81e/users/9ce64d49987c409093b3958c9cd9181c/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.653577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aabdf-3f90-4482-8b75-51a384fea7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=97def3b1b7c7427c873910ef0275b81e, user_id=9ce64d49987c409093b3958c9cd9181c, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.656349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aabdf-3f90-4482-8b75-51a384fea7ab admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.666654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 956/1906] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:33 2022] PUT /identity/v3/projects/97def3b1b7c7427c873910ef0275b81e/users/9ce64d49987c409093b3958c9cd9181c/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:33.678346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.678664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.678945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.713812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] MFA Rules not processed for user `9ce64d49987c409093b3958c9cd9181c`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:33.715086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:33.753430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 951/1907] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:33.756197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.816134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 957/1908] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:33.822172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.824096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:33.838127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb76c69-7dda-4151-a75d-89803695219c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.842866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.845151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.845151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.845151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.845151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.847075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.885974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25445d94-e8ca-4e4a-a963-5f3ccc043ddf None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.932537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 952/1909] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:33 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:33.935055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.935823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.936116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.936399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.936993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.947935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7d417-6088-4220-bedf-231a364c3df1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.953705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.954381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.954713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.954993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:33.955942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:33.957889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:33.975510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 953/1910] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:33.976577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:33.988797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11196691-e537-4b31-ab45-6eae8174de8b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:33.998451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:33.998451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:33.998451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:33.998451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] PATH_INFO: `/v3/projects/095f4d320bdf4d00893f206657bdf991/users/433043f6f0f04d24a9db4fd4aa12daed/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.017478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] RBAC: Authorizing `identity:create_grant(project_id=095f4d320bdf4d00893f206657bdf991, user_id=433043f6f0f04d24a9db4fd4aa12daed, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.023123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.028773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:34.046897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.048697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.060470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 958/1911] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:34.110543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 954/1912] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:33 2022] PUT /identity/v3/projects/095f4d320bdf4d00893f206657bdf991/users/433043f6f0f04d24a9db4fd4aa12daed/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:34.118469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:34.139739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e38084-cacf-4be9-81f5-4c6e1e947abc service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:34.246468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1bbc6-957c-4a50-a656-72a593878833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:34.249396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1bbc6-957c-4a50-a656-72a593878833 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:34.251348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1bbc6-957c-4a50-a656-72a593878833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:34.251348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1bbc6-957c-4a50-a656-72a593878833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.252631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1bbc6-957c-4a50-a656-72a593878833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.255090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1bbc6-957c-4a50-a656-72a593878833 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.286725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 959/1913] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:34.287548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:34.300903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8cc1d-6fde-431d-b724-e68068fd629a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:34.306204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:34.310456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:34.310456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:34.310456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] PATH_INFO: `/v3/projects/095f4d320bdf4d00893f206657bdf991/users/433043f6f0f04d24a9db4fd4aa12daed/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.310456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=095f4d320bdf4d00893f206657bdf991, user_id=433043f6f0f04d24a9db4fd4aa12daed, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.313181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.325420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 955/1914] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:34 2022] PUT /identity/v3/projects/095f4d320bdf4d00893f206657bdf991/users/433043f6f0f04d24a9db4fd4aa12daed/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:34.336309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:34.336605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:34.337166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.367410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] MFA Rules not processed for user `433043f6f0f04d24a9db4fd4aa12daed`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:34.368667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:34.394279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:34.449086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 960/1915] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:34.461983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:34.464049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:34.535154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497386ad-dfb4-4027-8645-0afb1989c06b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:34.539254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:34.540151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:34.540503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:34.540996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.551160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.552676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.563930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 956/1916] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:34.782496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:34.784907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:34.818395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:34.829297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8513b7-7749-445c-ab32-e77f046c52d3 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:34.844904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd6c97-446c-4919-a892-39e387bd7436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:34.845584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd6c97-446c-4919-a892-39e387bd7436 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:34.845874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd6c97-446c-4919-a892-39e387bd7436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:34.846157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd6c97-446c-4919-a892-39e387bd7436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.910939 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481f3b1-4640-452a-85b0-cf86ca129b22 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:34.921570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:34.923029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:34.923334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:34.923626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] PATH_INFO: `/v3/users/fd9054bbfbc44d4e9bf64686dfcd21c3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:34.927553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9054bbfbc44d4e9bf64686dfcd21c3)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.930678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.934137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd6c97-446c-4919-a892-39e387bd7436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:34.935807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd6c97-446c-4919-a892-39e387bd7436 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:34.949629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 957/1917] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:35.148267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:35.187166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9054bbfbc44d4e9bf64686dfcd21c3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.188095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9054bbfbc44d4e9bf64686dfcd21c3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.202979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9054bbfbc44d4e9bf64686dfcd21c3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.211206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9054bbfbc44d4e9bf64686dfcd21c3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.260853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:35.283189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9054bbfbc44d4e9bf64686dfcd21c3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.289846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 961/1918] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:34 2022] DELETE /identity/v3/users/fd9054bbfbc44d4e9bf64686dfcd21c3 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:35.294286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:35.294789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:35.294884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:35.316845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:35.319245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:35.409938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 958/1919] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:35.430705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:35.433273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:35.530078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba70e3-b634-4402-8ada-a42a742f2b4a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:35.534974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:35.536047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:35.536688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:35.537287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:35.549838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:35.552293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:35.561350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 962/1920] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:35.590613 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:35.594916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:35.630891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e46e21-9b8e-490e-a72c-8ae740cc5b7c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:35.650370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:35.653715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:35.654073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:35.654816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] PATH_INFO: `/v3/users/3c85aa9b1b1c49f7b848cec7b43bd6fc` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:35.658687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c85aa9b1b1c49f7b848cec7b43bd6fc)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:35.684749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:35.783799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:35.933659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:35.955921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c85aa9b1b1c49f7b848cec7b43bd6fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.968004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c85aa9b1b1c49f7b848cec7b43bd6fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.969848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c85aa9b1b1c49f7b848cec7b43bd6fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:35.989051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c85aa9b1b1c49f7b848cec7b43bd6fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.012863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:36.014315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.029265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02b72-af6a-476c-a315-35ca984632bd service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:36.034760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:36.035706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.036019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.036297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] PATH_INFO: `/v3/users/fa19d147eb1a4bc69097372d2737f9a0` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.037490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa19d147eb1a4bc69097372d2737f9a0)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:36.039538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:36.100794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c85aa9b1b1c49f7b848cec7b43bd6fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.102908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:36.113464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 959/1921] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:35 2022] DELETE /identity/v3/users/3c85aa9b1b1c49f7b848cec7b43bd6fc => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:36.113464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:36.114317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.186434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae6d2e-1c48-423b-980e-fffa99b96fda admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:36.191446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:36.192384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.192786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.193065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] PATH_INFO: `/v3/projects/1f18c992938d4a99beef4376d6a68565` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.194430 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f18c992938d4a99beef4376d6a68565)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:36.196225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:36.294374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa19d147eb1a4bc69097372d2737f9a0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.295037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa19d147eb1a4bc69097372d2737f9a0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.309150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa19d147eb1a4bc69097372d2737f9a0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.317344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa19d147eb1a4bc69097372d2737f9a0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.373829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f18c992938d4a99beef4376d6a68565', 'request_id': 'req-69aedb44-8349-4a13-83ab-5ed06155b66b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.457599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-16d5d608-8cc2-45aa-8c8b-e4fdbff7a29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa19d147eb1a4bc69097372d2737f9a0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.465039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 963/1922] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:36 2022] DELETE /identity/v3/users/fa19d147eb1a4bc69097372d2737f9a0 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:36.466504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b53f2-a9ad-4233-8cf5-195a812004f0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.466855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b53f2-a9ad-4233-8cf5-195a812004f0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.467137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b53f2-a9ad-4233-8cf5-195a812004f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.486458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-216b53f2-a9ad-4233-8cf5-195a812004f0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:36.487727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-216b53f2-a9ad-4233-8cf5-195a812004f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.548874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f18c992938d4a99beef4376d6a68565', 'request_id': 'req-69aedb44-8349-4a13-83ab-5ed06155b66b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.549920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 964/1923] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:36.551378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.554356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.554356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.557596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f18c992938d4a99beef4376d6a68565', 'request_id': 'req-69aedb44-8349-4a13-83ab-5ed06155b66b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.570345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] The token cache is being invalidate because project 1f18c992938d4a99beef4376d6a68565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:36.570637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.570872 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:36.571284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.572104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.572928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 960/1924] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:36 2022] DELETE /identity/v3/projects/1f18c992938d4a99beef4376d6a68565 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:36.573687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:36.575058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.580747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 965/1925] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:36.581658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:36.583067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.601627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed5867-fb3d-4888-a2d1-fbf32bd4c999 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:36.606497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:36.607654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.608505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.608505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] PATH_INFO: `/v3/users/babf4e70d3254767ae6288d35bca72e8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.609453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=babf4e70d3254767ae6288d35bca72e8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:36.611507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:36.616579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:36.647902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aedb44-8349-4a13-83ab-5ed06155b66b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:36.713333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:36.714046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.714344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.714630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.716591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.731606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:36.733019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:36.750856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 961/1926] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:36.752067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:36.753559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:36.764934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7f697-8f57-4a6c-a53d-b9a530f30710 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:36.770086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:36.770828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:36.771192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:36.771720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:36.792401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:36.794196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:36.807559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 962/1927] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:36.826766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babf4e70d3254767ae6288d35bca72e8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.827710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babf4e70d3254767ae6288d35bca72e8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.845648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babf4e70d3254767ae6288d35bca72e8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.854222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babf4e70d3254767ae6288d35bca72e8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.959649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babf4e70d3254767ae6288d35bca72e8'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:36.969848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 966/1928] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:36 2022] DELETE /identity/v3/users/babf4e70d3254767ae6288d35bca72e8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:36.983520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:37.405019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:37.406801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:37.421316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f394312-4f2e-41fa-9dd6-aebf2e7e4998 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:37.428604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:37.430150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:37.527429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae06a3e-fcf0-4647-90b2-4d4d9db748cb admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:37.527941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:37.528552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:37.529057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:37.529548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] PATH_INFO: `/v3/projects/45eaca6c57b049f6b11c604b2bcbb211` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:37.531028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45eaca6c57b049f6b11c604b2bcbb211)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:37.533931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:37.534771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:37.535750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:37.535978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:37.536252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] PATH_INFO: `/v3/projects/da3d5c0d75054fa383f9033bfb135e6d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:37.537781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3d5c0d75054fa383f9033bfb135e6d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:37.539848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:37.660548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3d5c0d75054fa383f9033bfb135e6d', 'request_id': 'req-edf3f19e-f85b-4b7d-a364-da26371c1ec9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.667629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3d5c0d75054fa383f9033bfb135e6d', 'request_id': 'req-edf3f19e-f85b-4b7d-a364-da26371c1ec9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.686810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:37.710902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45eaca6c57b049f6b11c604b2bcbb211', 'request_id': 'req-d234ea81-f560-44e5-bdf5-22240be97f20'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.781925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3d5c0d75054fa383f9033bfb135e6d', 'request_id': 'req-edf3f19e-f85b-4b7d-a364-da26371c1ec9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.794755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] The token cache is being invalidate because project da3d5c0d75054fa383f9033bfb135e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:37.795156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.796420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.801758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 967/1929] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:37 2022] DELETE /identity/v3/projects/da3d5c0d75054fa383f9033bfb135e6d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:37.802783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:37.804958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:37.880446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45eaca6c57b049f6b11c604b2bcbb211', 'request_id': 'req-d234ea81-f560-44e5-bdf5-22240be97f20'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.889236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45eaca6c57b049f6b11c604b2bcbb211', 'request_id': 'req-d234ea81-f560-44e5-bdf5-22240be97f20'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.912722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] The token cache is being invalidate because project 45eaca6c57b049f6b11c604b2bcbb211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:37.917892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.919577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:37.923181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 963/1930] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:37 2022] DELETE /identity/v3/projects/45eaca6c57b049f6b11c604b2bcbb211 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:37.926703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:37.927557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f19e-f85b-4b7d-a364-da26371c1ec9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:37.928410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:38.012583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:38.013291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.013604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.013874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] PATH_INFO: `/v3/users/9ce64d49987c409093b3958c9cd9181c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.015150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce64d49987c409093b3958c9cd9181c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:38.017218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:38.058907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234ea81-f560-44e5-bdf5-22240be97f20 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:38.064049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:38.064950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.065432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.065881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] PATH_INFO: `/v3/users/1f01d8f2e7d0479d8aa6660c178988b8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.067101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f01d8f2e7d0479d8aa6660c178988b8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:38.069102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:38.288769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce64d49987c409093b3958c9cd9181c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.289434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce64d49987c409093b3958c9cd9181c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.304756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce64d49987c409093b3958c9cd9181c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.312628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce64d49987c409093b3958c9cd9181c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.373873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f01d8f2e7d0479d8aa6660c178988b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.381328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f01d8f2e7d0479d8aa6660c178988b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.381986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f01d8f2e7d0479d8aa6660c178988b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.388254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce64d49987c409093b3958c9cd9181c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.395428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 968/1931] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:37 2022] DELETE /identity/v3/users/9ce64d49987c409093b3958c9cd9181c => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:38.398091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f01d8f2e7d0479d8aa6660c178988b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.400407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:38.404374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:38.496518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab2a74-0d28-4501-ad2e-5843a9c45e38 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:38.504966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:38.505876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.506170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.506444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] PATH_INFO: `/v3/projects/3599d124c77e4ab295d47ae98bb572fe` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.508149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3599d124c77e4ab295d47ae98bb572fe)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:38.510581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:38.525525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e514ede4-a4a3-4f18-82b9-62aeef57abd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f01d8f2e7d0479d8aa6660c178988b8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.533760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 964/1932] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:37 2022] DELETE /identity/v3/users/1f01d8f2e7d0479d8aa6660c178988b8 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:38.630130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3599d124c77e4ab295d47ae98bb572fe', 'request_id': 'req-c8450e80-108d-4f35-984d-f4abff5ead52'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.637741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3599d124c77e4ab295d47ae98bb572fe', 'request_id': 'req-c8450e80-108d-4f35-984d-f4abff5ead52'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.708548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3599d124c77e4ab295d47ae98bb572fe', 'request_id': 'req-c8450e80-108d-4f35-984d-f4abff5ead52'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.728775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] The token cache is being invalidate because project 3599d124c77e4ab295d47ae98bb572fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:38.729315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.730851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c8450e80-108d-4f35-984d-f4abff5ead52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:38.733171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 969/1933] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:38 2022] DELETE /identity/v3/projects/3599d124c77e4ab295d47ae98bb572fe => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:38.738413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.738910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.739028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.761879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:38.763447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:38.866697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 965/1934] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:38.876480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.876730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.877200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.897041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:38.898261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:38.909541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 970/1935] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:38.910723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:38.920767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74b1ad-4a1f-40a6-8eef-6193f6dccae2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:38.928410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:38.928410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.928410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.928410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.928749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:38.928749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:38.930404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:38.951705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 966/1936] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:28:38.951900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:38.962423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6477c9-f9e8-42e0-bf17-5e5d0b902149 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:38.967134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:38.971619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:38.974808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:38.987716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:38.989415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:38.991552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:38.992819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:38.993748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:38.997871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:38.999504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885266-356f-44a4-b87f-b6aeafa08ea5 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.025648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.026894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.030440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.038229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] PATH_INFO: `/v3/projects/97def3b1b7c7427c873910ef0275b81e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.040607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=97def3b1b7c7427c873910ef0275b81e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.047486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.138584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 971/1937] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:38 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:39.144392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.209877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97def3b1b7c7427c873910ef0275b81e', 'request_id': 'req-9f09c343-2909-4a11-966d-a9af5d9d0a7d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.217322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c7ea4-a71f-4268-87ed-38f97a032b8e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.278008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae7936-117b-428b-8155-af0d73112aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.278837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7936-117b-428b-8155-af0d73112aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.279227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7936-117b-428b-8155-af0d73112aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.279636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae7936-117b-428b-8155-af0d73112aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.280437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7936-117b-428b-8155-af0d73112aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.282527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae7936-117b-428b-8155-af0d73112aa8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.303132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97def3b1b7c7427c873910ef0275b81e', 'request_id': 'req-9f09c343-2909-4a11-966d-a9af5d9d0a7d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.313660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97def3b1b7c7427c873910ef0275b81e', 'request_id': 'req-9f09c343-2909-4a11-966d-a9af5d9d0a7d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.327385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] The token cache is being invalidate because project 97def3b1b7c7427c873910ef0275b81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:39.328385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.329995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.332274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 967/1938] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:38 2022] DELETE /identity/v3/projects/97def3b1b7c7427c873910ef0275b81e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:39.332868 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.334263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.349587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09c343-2909-4a11-966d-a9af5d9d0a7d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.355542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.356710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.357002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.357288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] PATH_INFO: `/v3/projects/49152295ab4446769929ce8408e40d17` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.358800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=49152295ab4446769929ce8408e40d17)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.361414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.379515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 972/1939] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:39.381924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.384018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.384518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.404578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:39.405788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.414745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 973/1940] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:39.415557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.416790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.429259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7a50b-4e61-46c6-9465-d41769314f62 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.433943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.434638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.434922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.435197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.435957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.437676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.447763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 974/1941] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:39.448574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.451337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.466099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8e02-c18f-402f-a9a5-867bec2f2fec admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.472333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.472823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.473097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.473379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] PATH_INFO: `/v3/users/35c9271ba32e4197b96ab6ed3eeb9d00` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.474628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c9271ba32e4197b96ab6ed3eeb9d00)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.476615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.505519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49152295ab4446769929ce8408e40d17', 'request_id': 'req-8c994d8d-938f-46e6-9020-58732ba45fcd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.599616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49152295ab4446769929ce8408e40d17', 'request_id': 'req-8c994d8d-938f-46e6-9020-58732ba45fcd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.605604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49152295ab4446769929ce8408e40d17', 'request_id': 'req-8c994d8d-938f-46e6-9020-58732ba45fcd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.617358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] The token cache is being invalidate because project 49152295ab4446769929ce8408e40d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:39.617784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.618474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8c994d8d-938f-46e6-9020-58732ba45fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.620145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 968/1942] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:39 2022] DELETE /identity/v3/projects/49152295ab4446769929ce8408e40d17 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:39.621557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.621824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.622104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.643288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c9271ba32e4197b96ab6ed3eeb9d00'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.644678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c9271ba32e4197b96ab6ed3eeb9d00'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.646303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:39.647562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.662893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c9271ba32e4197b96ab6ed3eeb9d00'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.670889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c9271ba32e4197b96ab6ed3eeb9d00'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.714722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:39.725808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 969/1943] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:39.726943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.728583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.743958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead84d7-ea8d-463b-ac77-1ba6f7149c2e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.751224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.752555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.758825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.758825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.759668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2435ef3c-dea5-46be-a252-96fd6bebd066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c9271ba32e4197b96ab6ed3eeb9d00'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:39.759773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.759773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.762321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 975/1944] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:39 2022] DELETE /identity/v3/users/35c9271ba32e4197b96ab6ed3eeb9d00 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:39.763937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608bde-1a01-420b-a2f7-a7f431445a48 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.764555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608bde-1a01-420b-a2f7-a7f431445a48 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.765109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608bde-1a01-420b-a2f7-a7f431445a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.771042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 970/1945] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:39.772065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.786964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2f980-f5ce-4ba3-bd72-bae90ad6e13b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.789278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-84608bde-1a01-420b-a2f7-a7f431445a48 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:39.790989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-84608bde-1a01-420b-a2f7-a7f431445a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.849000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.849744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.850028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.850337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.851087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:39.851435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.853271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.860699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 976/1946] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:39.862504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea700-d438-4b67-82d1-5b1579db5565 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.862819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea700-d438-4b67-82d1-5b1579db5565 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.863094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea700-d438-4b67-82d1-5b1579db5565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.873316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 971/1947] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:39.874440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.886997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182fc7f-9cd2-42fb-9a27-c3b7d330e58e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.888496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d40ea700-d438-4b67-82d1-5b1579db5565 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:39.889684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d40ea700-d438-4b67-82d1-5b1579db5565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.892733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.893791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.894225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.894652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] PATH_INFO: `/v3/projects/52bd1ed3d31141cfb9cd4eac2499194d/users/3f909a7a8b6340a2b062043cd53c8c84/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.900781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 977/1948] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:39.905466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.905764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.906093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.916139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bd1ed3d31141cfb9cd4eac2499194d, user_id=3f909a7a8b6340a2b062043cd53c8c84, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.919016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.929708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:39.930761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:39.940115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 978/1949] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:39.941292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.950794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a98d9-1712-434f-ad0d-ae4b43c3c342 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:39.956337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:39.956999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:39.957287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:39.957592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:39.958282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:39.960340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:39.983623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 972/1950] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:39 2022] PUT /identity/v3/projects/52bd1ed3d31141cfb9cd4eac2499194d/users/3f909a7a8b6340a2b062043cd53c8c84/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:39.985576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:39.988503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:40.063783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f3fb0-5370-44df-bf48-532cf0387b2f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.067680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fe76d-1fe1-4f79-8175-53477f7e078d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.068779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fe76d-1fe1-4f79-8175-53477f7e078d service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.069104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fe76d-1fe1-4f79-8175-53477f7e078d service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.069393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fe76d-1fe1-4f79-8175-53477f7e078d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.087148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 979/1951] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:39 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:40.088067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.145004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fe76d-1fe1-4f79-8175-53477f7e078d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.148039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fe76d-1fe1-4f79-8175-53477f7e078d service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.168590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 973/1952] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:40.171610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b59f51-98a7-4658-8502-d527409cfdaf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.173192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.173499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.174567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.180139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.180974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.181268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.182269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.182269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:40.182638 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.184453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.202223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 980/1953] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:40.203095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.210263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] MFA Rules not processed for user `3f909a7a8b6340a2b062043cd53c8c84`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:40.212010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:40.216171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8189-6994-4d08-a5f6-9faf9144d569 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.222958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.223746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.224547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.225054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.226317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.228447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.313547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 974/1954] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:40.318125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.331339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dff54-3c7a-46ca-8eac-e2696be576df None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.337314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.338092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.338392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.338715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.339753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.345118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.351798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 981/1955] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:40 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:40.356876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.372375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78b2f6-36e3-482f-b7b7-a688d9bcfc4d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.378455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.379142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.379471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.380047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.381127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.386015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.446585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 975/1956] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:40 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:40.447691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.521314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552599f-eb92-420e-b363-de7b39cd14a0 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.526873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.527663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.527946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.528227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.528974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.530020 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 982/1957] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:40 2022] POST /identity/v3/projects => generated 367 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:40.530785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.531427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.560087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 976/1958] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:40.561055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.571206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358dc2d-fbae-4bd8-b69b-745e5e177883 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.614389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8b66d-afa5-4287-8609-4a654cabee90 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.619997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.621184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.621434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.621766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.622564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.627031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.651081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.651999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.652351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.652682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.657547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.657547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.674669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:40.706612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 983/1959] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:40 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:40.706612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.716469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3abb7-7156-48ed-a1c1-a24c7dba1dcb admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.722932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.723662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.723951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.724272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.725008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.731039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.733514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 977/1960] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:40 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:40.738088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 984/1961] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:40.739125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.740212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.753306 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41cf68-a68e-4eaf-841c-a4c2073cb69c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.756612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7a71-f751-4284-a420-7f5524871501 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.758717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.759540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.759892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.760327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.761268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.761716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.762421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.762710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.762988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.763154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.763763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.765392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.776270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 985/1962] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:40.777208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.780980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 978/1963] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:40.781846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.789258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a16f7-2dad-4f42-98fa-b2d9c025bcaa admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.791488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e6a95-7f9b-42a7-9f2f-307c86b990c4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:40.795772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.796661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.796965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.797257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] PATH_INFO: `/v3/projects/3284a94a4c3b4f2a96b8e266902d14e9/users/9b1f72dd7ab6468c9600ce8e4e2f5dc0/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.798242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:40.799192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:40.799580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:40.799928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] PATH_INFO: `/v3/projects/ebd104c85e0242b2872c540ad62d5355/users/3531399ff41a4b91a4852d608b65188c/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:40.811608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3284a94a4c3b4f2a96b8e266902d14e9, user_id=9b1f72dd7ab6468c9600ce8e4e2f5dc0, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.814795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.815567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd104c85e0242b2872c540ad62d5355, user_id=3531399ff41a4b91a4852d608b65188c, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:40.818218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:40.893599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 986/1964] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:40 2022] PUT /identity/v3/projects/3284a94a4c3b4f2a96b8e266902d14e9/users/9b1f72dd7ab6468c9600ce8e4e2f5dc0/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:40.894510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.947125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 979/1965] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:40 2022] PUT /identity/v3/projects/ebd104c85e0242b2872c540ad62d5355/users/3531399ff41a4b91a4852d608b65188c/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:40.947125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:40.947125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:40.962175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad275a41-794e-4f15-bd02-e02a94642394 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.010175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeedb83-c519-4612-894a-d94e18f5e278 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.014803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.015705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.016586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.016870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] PATH_INFO: `/v3/projects/09966bf372384c24a7f8087c322dbc8f` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.018052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09966bf372384c24a7f8087c322dbc8f)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.019440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8ac4-0541-4b8d-be1a-bdd38f2ccd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.020073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.020178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8ac4-0541-4b8d-be1a-bdd38f2ccd36 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.020378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8ac4-0541-4b8d-be1a-bdd38f2ccd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.020654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8ac4-0541-4b8d-be1a-bdd38f2ccd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.021410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8ac4-0541-4b8d-be1a-bdd38f2ccd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.022990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8ac4-0541-4b8d-be1a-bdd38f2ccd36 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.033325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 987/1966] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:41.034637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.034929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.035211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.066153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] MFA Rules not processed for user `9b1f72dd7ab6468c9600ce8e4e2f5dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:41.066603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.141505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 988/1967] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:41.141505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.150475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04aecc-efae-47fa-916f-07f5ac3dc7df None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.153627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09966bf372384c24a7f8087c322dbc8f', 'request_id': 'req-c061d736-f108-4cdd-9f99-212bd18cde65'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.210188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.211079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.211513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.211892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.212805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.214550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.223434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:41.226327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 989/1968] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:41.227178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.237349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3d2e5-70f5-40a0-b788-38c939c6cee0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.242077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.242952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.243241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.243547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] PATH_INFO: `/v3/projects/2e28f09e5fb24b07a592c382efba3ef5/users/81620733d85e415f855592199bd81a49/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.262058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e28f09e5fb24b07a592c382efba3ef5, user_id=81620733d85e415f855592199bd81a49, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.264718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.268449 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09966bf372384c24a7f8087c322dbc8f', 'request_id': 'req-c061d736-f108-4cdd-9f99-212bd18cde65'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.276108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09966bf372384c24a7f8087c322dbc8f', 'request_id': 'req-c061d736-f108-4cdd-9f99-212bd18cde65'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.288049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] The token cache is being invalidate because project 09966bf372384c24a7f8087c322dbc8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:41.288433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.289162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.290835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 980/1969] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:40 2022] DELETE /identity/v3/projects/09966bf372384c24a7f8087c322dbc8f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:41.291682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.292971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.308297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061d736-f108-4cdd-9f99-212bd18cde65 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.317396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.318138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.318425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.318723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.319464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.321460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.348868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 990/1970] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:41 2022] PUT /identity/v3/projects/2e28f09e5fb24b07a592c382efba3ef5/users/81620733d85e415f855592199bd81a49/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:41.349997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.351873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.430566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4943e-0382-4479-81cb-71d10b4ac31f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.442086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce83107-cd4f-49bc-8114-5438562461b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.443004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce83107-cd4f-49bc-8114-5438562461b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.443295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce83107-cd4f-49bc-8114-5438562461b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.443603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce83107-cd4f-49bc-8114-5438562461b2 admin admin] PATH_INFO: `/v3/projects/ebd104c85e0242b2872c540ad62d5355/users/3531399ff41a4b91a4852d608b65188c/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.448772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce83107-cd4f-49bc-8114-5438562461b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd104c85e0242b2872c540ad62d5355, user_id=3531399ff41a4b91a4852d608b65188c, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.449453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 981/1971] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:41 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:41.450534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.451585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce83107-cd4f-49bc-8114-5438562461b2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.457222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.471317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 991/1972] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:41 2022] PUT /identity/v3/projects/ebd104c85e0242b2872c540ad62d5355/users/3531399ff41a4b91a4852d608b65188c/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:41.472524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.472817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.473095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.525813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] MFA Rules not processed for user `81620733d85e415f855592199bd81a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:41.527316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.536497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c8ee3-b0b5-4df8-ae68-f492ccbfe59d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.602436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.603655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.604192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.604655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] PATH_INFO: `/v3/users/433043f6f0f04d24a9db4fd4aa12daed` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.605915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] RBAC: Authorizing `identity:delete_user(user_id=433043f6f0f04d24a9db4fd4aa12daed)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.608123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 992/1973] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:41.608246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.609235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.621374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785bd80-81ff-4a5b-9afb-c74b127e3ff1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.628543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc545e-2973-4d87-844a-808fa94f904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.630289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc545e-2973-4d87-844a-808fa94f904f admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.630828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc545e-2973-4d87-844a-808fa94f904f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.632060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc545e-2973-4d87-844a-808fa94f904f admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.632925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc545e-2973-4d87-844a-808fa94f904f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.635895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc545e-2973-4d87-844a-808fa94f904f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.800714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 993/1974] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:41 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:41.802381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.802626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.802916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.838285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433043f6f0f04d24a9db4fd4aa12daed'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.839292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] MFA Rules not processed for user `3531399ff41a4b91a4852d608b65188c`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:41.840646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.848039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433043f6f0f04d24a9db4fd4aa12daed'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.848703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433043f6f0f04d24a9db4fd4aa12daed'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.871589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433043f6f0f04d24a9db4fd4aa12daed'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.915122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 994/1975] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:41.916092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.917562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:41.932329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db170f-6570-46a9-9e48-2bec4338b6fc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:41.937554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433043f6f0f04d24a9db4fd4aa12daed'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:41.938684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:41.939524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:41.939961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:41.940116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:41.940797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:41.942728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:41.946356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 982/1976] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:41 2022] DELETE /identity/v3/users/433043f6f0f04d24a9db4fd4aa12daed => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:41.946356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:41.947953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.018772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893558-63fd-4a68-bd8a-ba5754df46df admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.023505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.024186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.024468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.024734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.025522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.028319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.041221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 995/1977] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:41 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:42.041392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 983/1978] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:42.042240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.043263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.046170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.099895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:42.109120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdcde3-b708-43ae-ab44-4afd54250050 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.119978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125d0ad-cb77-48bb-ae4d-3c3e84f78b32 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.124704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.125394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.125793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.126079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.127027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.129184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.161893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.162562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.162847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.163128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.194194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 996/1979] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:42 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:42.194194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.205035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187f366-7053-4876-aa14-42dc13de43ac admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.209861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.210955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.211239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.211592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] PATH_INFO: `/v3/projects/a6ac7833edb94f5c8f9411ebc95064aa/users/8eab6df66a0649eaa0c5f661fd7b18dd/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.226307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.227793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.228474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ac7833edb94f5c8f9411ebc95064aa, user_id=8eab6df66a0649eaa0c5f661fd7b18dd, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.231565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.238172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 984/1980] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:42.239014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.249288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcdc06-0666-43af-b8be-7e3d40b8d279 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.257258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.257655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.257940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.258219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.259205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.260661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.271955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 985/1981] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:42.275931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.285531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2832bf-8163-4fcf-8dd4-9da38f979185 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.290446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.291542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.291770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.292047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] PATH_INFO: `/v3/projects/94abbff1ece746cea114f94a4bccc10c/users/8fdd96b07e6e47009549f7029a354d89/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.304233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 997/1982] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:42 2022] PUT /identity/v3/projects/a6ac7833edb94f5c8f9411ebc95064aa/users/8eab6df66a0649eaa0c5f661fd7b18dd/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:42.309215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94abbff1ece746cea114f94a4bccc10c, user_id=8fdd96b07e6e47009549f7029a354d89, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.310938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.312033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.389639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 986/1983] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:42 2022] PUT /identity/v3/projects/94abbff1ece746cea114f94a4bccc10c/users/8fdd96b07e6e47009549f7029a354d89/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:42.395738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.399758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.407468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bed70-b6c9-4385-8e4b-c3e5b3ffcfcb admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.440306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:42.485752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ab333-a525-4bed-a8bb-3c7c4eff2f1a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.490140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.491147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.491367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.491658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.494946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.505783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.507533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.508255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.508754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.509699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.511821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.526127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 998/1984] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:42.528076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.540759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce061403-8403-4ddc-80b2-2c743e524de4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.547073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.549195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.549694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.550160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.551100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.553253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.569257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 999/1985] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:42.570149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.575410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.576949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.581392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca1700-19b8-4cb6-873e-ea53d8d4aebb admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.586351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa92da4-b853-4416-ae6f-0e9bd339a4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.587246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92da4-b853-4416-ae6f-0e9bd339a4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.588687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92da4-b853-4416-ae6f-0e9bd339a4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.588977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92da4-b853-4416-ae6f-0e9bd339a4ec admin admin] PATH_INFO: `/v3/projects/a6ac7833edb94f5c8f9411ebc95064aa/users/8eab6df66a0649eaa0c5f661fd7b18dd/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.589431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 987/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:42.590342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.593801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa92da4-b853-4416-ae6f-0e9bd339a4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ac7833edb94f5c8f9411ebc95064aa, user_id=8eab6df66a0649eaa0c5f661fd7b18dd, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.597428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa92da4-b853-4416-ae6f-0e9bd339a4ec admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.616405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe13d0-de3a-4735-84fa-8dd7ed8d5b35 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.617312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1000/1987] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:42 2022] PUT /identity/v3/projects/a6ac7833edb94f5c8f9411ebc95064aa/users/8eab6df66a0649eaa0c5f661fd7b18dd/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:42.632295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.632295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.632295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.680284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] MFA Rules not processed for user `8eab6df66a0649eaa0c5f661fd7b18dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:42.682833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.698563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3bd4b-604a-463f-96c5-033f5b18fe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.699484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3bd4b-604a-463f-96c5-033f5b18fe7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.699870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3bd4b-604a-463f-96c5-033f5b18fe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.700867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3bd4b-604a-463f-96c5-033f5b18fe7a admin admin] PATH_INFO: `/v3/projects/94abbff1ece746cea114f94a4bccc10c/users/8fdd96b07e6e47009549f7029a354d89/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.702409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3bd4b-604a-463f-96c5-033f5b18fe7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94abbff1ece746cea114f94a4bccc10c, user_id=8fdd96b07e6e47009549f7029a354d89, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.705941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3bd4b-604a-463f-96c5-033f5b18fe7a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.725511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 988/1988] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:42 2022] PUT /identity/v3/projects/94abbff1ece746cea114f94a4bccc10c/users/8fdd96b07e6e47009549f7029a354d89/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:42.736994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.737526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.738013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.774054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] MFA Rules not processed for user `8fdd96b07e6e47009549f7029a354d89`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:42.775331 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.803232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1001/1989] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:42.804522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.806661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:42.844692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494df307-f3cd-4cc8-b7e9-27571703492b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.884286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 989/1990] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:42.884286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:42.903798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:42.915741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.916704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.917104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.917452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] PATH_INFO: `/v3/projects/095f4d320bdf4d00893f206657bdf991` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:42.918773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095f4d320bdf4d00893f206657bdf991)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:42.920927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:42.944814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da5295-7ea0-4137-85ce-0f3faa23e5b0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:42.948707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:42.949541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:42.949833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:42.950118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.014371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.016521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.026111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 990/1991] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:43.074654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095f4d320bdf4d00893f206657bdf991', 'request_id': 'req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:43.081004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095f4d320bdf4d00893f206657bdf991', 'request_id': 'req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:43.216789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.225156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a703b-ec03-45d7-bf24-5b3c48d0c38c service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.262483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095f4d320bdf4d00893f206657bdf991', 'request_id': 'req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:43.276961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] The token cache is being invalidate because project 095f4d320bdf4d00893f206657bdf991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:43.277259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:43.277661 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.277909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2b1cd72c-3a44-4e5a-b46e-9a0c1154e933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:43.278324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.278609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.278919 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.279472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1002/1992] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:42 2022] DELETE /identity/v3/projects/095f4d320bdf4d00893f206657bdf991 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:43.280942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:43.284213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.284481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.284759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.305130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:43.306405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:43.346770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.348190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b7f8-0fc3-43aa-8f24-6e31d0eabb58 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.357963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 991/1993] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:43.369529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1003/1994] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:43.378720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.379013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.379296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.407830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:43.409049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:43.417398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 992/1995] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:43.421678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.432490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba5e73-dbac-4b68-8f8a-05fedf08f577 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.434252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:43.437096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.437807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.438958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.439075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.439943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:43.440385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.442000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.454654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1004/1996] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:43.458700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.473854 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a1689-bc3d-4fab-a1b1-ae5f99a6ed59 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.478385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.479412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.479700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.480004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.480709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.483233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.486319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.488149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:43.501962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411abf1d-33b5-406a-b57a-c7a5d2d4dc1d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.506992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.506992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.507768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.508377 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.510460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:43.582885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 993/1997] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:43 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:43.587716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.590438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.592047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.602331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1005/1998] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:43.661996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35d82c-7c50-4c33-88c5-83630d567631 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.667062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.667895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.668309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.668664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.669633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.671718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.777043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 994/1999] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:43 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:43.781163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.791761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dac9ad-d0ef-422d-bea0-cef0b6c51db1 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.796250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.796905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.797239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.797542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.798309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.799941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.809824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1006/2000] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:43.814178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.824232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d9559-1ca6-4b82-945a-11abf655ee2c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.829557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacbc48-336f-43b7-ace3-700a8c18dc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.830225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacbc48-336f-43b7-ace3-700a8c18dc98 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.830514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacbc48-336f-43b7-ace3-700a8c18dc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.830794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacbc48-336f-43b7-ace3-700a8c18dc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.831592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacbc48-336f-43b7-ace3-700a8c18dc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.833201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacbc48-336f-43b7-ace3-700a8c18dc98 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.844952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 995/2001] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:43.848838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:43.858396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:43.868639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9af5-0a33-43fe-9d9a-e4c22ae438b6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:43.873289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:43.874260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:43.874458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:43.874704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] PATH_INFO: `/v3/projects/e40a39858afa41bb96f902c6178d44f3/users/4d3f819f1a8d4db99774cef9d9a0b493/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:43.890775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40a39858afa41bb96f902c6178d44f3, user_id=4d3f819f1a8d4db99774cef9d9a0b493, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:43.895507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:43.941016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:44.003390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1007/2002] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:43 2022] PUT /identity/v3/projects/e40a39858afa41bb96f902c6178d44f3/users/4d3f819f1a8d4db99774cef9d9a0b493/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:44.013924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.014365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.014655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.043168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] MFA Rules not processed for user `4d3f819f1a8d4db99774cef9d9a0b493`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:44.046584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:44.116776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 996/2003] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:44.121327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:44.202341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176f51-2e03-4a2f-889e-77c0032f05b0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:44.207844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:44.208568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.208860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.209145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.209924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:44.211908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:44.288283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1008/2004] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:44 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:44.290377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:44.305029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90921c0f-2d3f-46ce-a273-32f66f0da0f7 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:44.369053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:44.369677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.369966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.370251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.370948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:44.372910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:44.455332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 997/2005] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:44 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:44.462973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:44.473712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedb829-fc10-4af6-9650-8a771ccfe333 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:44.478331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:44.478982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.479268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.479579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.480304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:44.481911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:44.493289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1009/2006] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:44.497390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:44.508117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a661-ee09-4c86-8212-183dd1484ee5 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:44.512977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:44.513867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.514149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.514423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] PATH_INFO: `/v3/projects/d5e72842c30a4d8e976f03a16ebac257/users/f4f1da3d6537498a8c65f368a6c6dc5d/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.530302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e72842c30a4d8e976f03a16ebac257, user_id=f4f1da3d6537498a8c65f368a6c6dc5d, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:44.534105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:44.595082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:44.631779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 998/2007] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:44 2022] PUT /identity/v3/projects/d5e72842c30a4d8e976f03a16ebac257/users/f4f1da3d6537498a8c65f368a6c6dc5d/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:44.634255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:44.636446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:44.647487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c2ba-e7b4-4d6d-9e3d-8fcbcd2efc29 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:44.704607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5532a61-1ec5-4273-ab40-d90d479c61ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:44.705276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5532a61-1ec5-4273-ab40-d90d479c61ed admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.705539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5532a61-1ec5-4273-ab40-d90d479c61ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.710786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5532a61-1ec5-4273-ab40-d90d479c61ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.710786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5532a61-1ec5-4273-ab40-d90d479c61ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:44.710786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5532a61-1ec5-4273-ab40-d90d479c61ed admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:44.725097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1010/2008] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:44.729867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:44.749062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e63c47-d1b1-4206-b60a-fe0b04f02100 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:44.755464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:44.756506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.756695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.756974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] PATH_INFO: `/v3/projects/d5e72842c30a4d8e976f03a16ebac257/users/f4f1da3d6537498a8c65f368a6c6dc5d/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.758986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e72842c30a4d8e976f03a16ebac257, user_id=f4f1da3d6537498a8c65f368a6c6dc5d, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:44.761780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:44.785638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 999/2009] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:44 2022] PUT /identity/v3/projects/d5e72842c30a4d8e976f03a16ebac257/users/f4f1da3d6537498a8c65f368a6c6dc5d/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:44.798107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:44.798107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:44.798107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:44.827788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] MFA Rules not processed for user `f4f1da3d6537498a8c65f368a6c6dc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:44.829030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:44.906808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1011/2010] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:45.026243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:45.090453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:45.092039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:45.099825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068695d-b8db-4adf-af2b-5099f138bccf admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:45.103414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1634d-68a3-4fce-a6ad-99c2fe69346b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:45.104067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1634d-68a3-4fce-a6ad-99c2fe69346b service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:45.104349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1634d-68a3-4fce-a6ad-99c2fe69346b service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:45.104899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1634d-68a3-4fce-a6ad-99c2fe69346b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:45.160442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ae9c9-8b4b-433d-8bf2-d08a1d973d73 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:45.164867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1634d-68a3-4fce-a6ad-99c2fe69346b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:45.165202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:45.166107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:45.166387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1634d-68a3-4fce-a6ad-99c2fe69346b service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:45.166493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:45.166666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] PATH_INFO: `/v3/users/81620733d85e415f855592199bd81a49` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:45.168065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81620733d85e415f855592199bd81a49)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:45.169991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:45.176214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1000/2011] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:45.347716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:45.352589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81620733d85e415f855592199bd81a49'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:45.353315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81620733d85e415f855592199bd81a49'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:45.366620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81620733d85e415f855592199bd81a49'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:45.374200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81620733d85e415f855592199bd81a49'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:45.486118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81620733d85e415f855592199bd81a49'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:45.492493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1012/2012] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:45 2022] DELETE /identity/v3/users/81620733d85e415f855592199bd81a49 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:45.497071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:45.497365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:45.497705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:45.514802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:45.516042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:45.573973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1001/2013] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:45.584076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:45.589721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:45.645173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400e716-5949-4874-ba0d-08cc4baa6b0d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:45.648804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:45.649451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:45.649741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:45.650019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:45.659695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:45.664240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:45.670344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1013/2014] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:45.712341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:45.912481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:46.131471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:46.133020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:46.144965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa632f-4414-4f1a-9b94-22c611b00c22 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:46.150616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:46.151940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:46.152420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:46.152732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] PATH_INFO: `/v3/projects/2e28f09e5fb24b07a592c382efba3ef5` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:46.154106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e28f09e5fb24b07a592c382efba3ef5)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:46.156061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:46.292053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e28f09e5fb24b07a592c382efba3ef5', 'request_id': 'req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.371446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e28f09e5fb24b07a592c382efba3ef5', 'request_id': 'req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.378047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e28f09e5fb24b07a592c382efba3ef5', 'request_id': 'req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.389728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] The token cache is being invalidate because project 2e28f09e5fb24b07a592c382efba3ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:46.390017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.390688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.392934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1002/2015] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:46 2022] DELETE /identity/v3/projects/2e28f09e5fb24b07a592c382efba3ef5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:46.398002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:46.410789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83441e-a22c-4b98-a050-35e9ab28b100 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:46.465813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:46.466835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:46.467216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:46.467619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] PATH_INFO: `/v3/users/8fdd96b07e6e47009549f7029a354d89` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:46.468949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdd96b07e6e47009549f7029a354d89)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:46.471576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:46.647637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:46.684156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:46.702067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdd96b07e6e47009549f7029a354d89'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.702826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdd96b07e6e47009549f7029a354d89'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.716265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdd96b07e6e47009549f7029a354d89'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.722883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdd96b07e6e47009549f7029a354d89'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.848846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a22a98b7-9929-47db-af92-188f98ffecad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdd96b07e6e47009549f7029a354d89'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:46.855836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1014/2016] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:46 2022] DELETE /identity/v3/users/8fdd96b07e6e47009549f7029a354d89 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:47.461492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:47.476336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:47.479105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:47.496183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f039b4d-022d-4d84-aab1-cd4f0dba8c72 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:47.571850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:47.572785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:47.573055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:47.573337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] PATH_INFO: `/v3/projects/94abbff1ece746cea114f94a4bccc10c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:47.574556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94abbff1ece746cea114f94a4bccc10c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:47.576547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:47.686812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:47.708768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94abbff1ece746cea114f94a4bccc10c', 'request_id': 'req-c50ee9d8-2cda-4f74-8785-a052c8cc9565'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:47.776770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94abbff1ece746cea114f94a4bccc10c', 'request_id': 'req-c50ee9d8-2cda-4f74-8785-a052c8cc9565'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:47.783357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94abbff1ece746cea114f94a4bccc10c', 'request_id': 'req-c50ee9d8-2cda-4f74-8785-a052c8cc9565'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:47.795973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] The token cache is being invalidate because project 94abbff1ece746cea114f94a4bccc10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:47.796252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:47.797745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:47.799570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1003/2017] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:47 2022] DELETE /identity/v3/projects/94abbff1ece746cea114f94a4bccc10c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:47.804606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdd7c-cae4-4fd4-b97d-64c4f7d1fa92 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:47.804972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdd7c-cae4-4fd4-b97d-64c4f7d1fa92 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:47.805346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdd7c-cae4-4fd4-b97d-64c4f7d1fa92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:47.824667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3c9cdd7c-cae4-4fd4-b97d-64c4f7d1fa92 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:47.825939 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3c9cdd7c-cae4-4fd4-b97d-64c4f7d1fa92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:47.852239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:47.853712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:47.910504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1015/2018] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:47.918666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:47.918953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:47.919233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:47.928710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ee9d8-2cda-4f74-8785-a052c8cc9565 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:47.933444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:47.934478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:47.934763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:47.935038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] PATH_INFO: `/v3/users/6b229a040ed74de0a848cdd9970058c8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:47.936393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b229a040ed74de0a848cdd9970058c8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:47.939947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:47.948254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:47.949553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:47.958312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1016/2019] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:47.962522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:47.971846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277daf8-3d7c-4c99-8353-940634d09762 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:47.976982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:47.977685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:47.977975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:47.978257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:47.979042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:47.979454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:47.981175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:47.994180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1017/2020] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:28:47.998346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.008694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aed595-244b-45fd-9679-c2c9f64ed263 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.013420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.014043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.014324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.014601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.015276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:48.017210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:48.149977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1018/2021] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:47 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:48.153871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.186109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b229a040ed74de0a848cdd9970058c8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.193153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b229a040ed74de0a848cdd9970058c8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.193990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b229a040ed74de0a848cdd9970058c8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.206606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b229a040ed74de0a848cdd9970058c8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.230224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa515b97-9b2f-4a55-887b-1ee7199fe0c6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.236387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.237240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.237669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.238089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.238925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:48.240884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:48.319719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b229a040ed74de0a848cdd9970058c8'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.326705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1004/2022] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:47 2022] DELETE /identity/v3/users/6b229a040ed74de0a848cdd9970058c8 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:48.384287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1019/2023] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:48 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:48.388621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.390647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:48.490630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee537e-9741-4e79-8831-38235f46e14d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.501114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.503165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.503601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.512972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.512972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:48.512972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:48.517647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1005/2024] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:48.521658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.534210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f137a-41e0-455c-9af3-ec7d3eabf1e5 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.540355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.541033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.541345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.541593 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.542358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:48.544071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:48.560054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1020/2025] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:48.562254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.573481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac09477-2fd5-4bef-97b9-718b4572c959 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.579182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1151ea-0271-4e3e-a6c5-d7bc04cff59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.580351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1151ea-0271-4e3e-a6c5-d7bc04cff59e admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.580638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1151ea-0271-4e3e-a6c5-d7bc04cff59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.580918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1151ea-0271-4e3e-a6c5-d7bc04cff59e admin admin] PATH_INFO: `/v3/projects/e0d7f979a0854c3eac771832da27c37e/users/8d6dec4577db46fbaab5d0557345e1a9/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.596690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1151ea-0271-4e3e-a6c5-d7bc04cff59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d7f979a0854c3eac771832da27c37e, user_id=8d6dec4577db46fbaab5d0557345e1a9, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:48.599809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1151ea-0271-4e3e-a6c5-d7bc04cff59e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:48.626582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.631239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:48.647922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42faac-701a-4c22-9e7b-3269150f131a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.655685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.655685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.655685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.655948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] PATH_INFO: `/v3/projects/a6235b2a54ef4164aa1e1106e819bc17` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.657732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6235b2a54ef4164aa1e1106e819bc17)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:48.659625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:48.713129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1006/2026] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:48 2022] PUT /identity/v3/projects/e0d7f979a0854c3eac771832da27c37e/users/8d6dec4577db46fbaab5d0557345e1a9/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:48.723237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.723624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.723984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:48.753301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] MFA Rules not processed for user `8d6dec4577db46fbaab5d0557345e1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:48.754517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:48.803795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6235b2a54ef4164aa1e1106e819bc17', 'request_id': 'req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.809310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6235b2a54ef4164aa1e1106e819bc17', 'request_id': 'req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.844610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1007/2027] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:48.845538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.846859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:48.898091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6235b2a54ef4164aa1e1106e819bc17', 'request_id': 'req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.926257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63658d-c92a-4ada-b50d-f34673bc38d8 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:48.927010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] The token cache is being invalidate because project a6235b2a54ef4164aa1e1106e819bc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:48.936914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.947682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:48.947682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1021/2028] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:48 2022] DELETE /identity/v3/projects/a6235b2a54ef4164aa1e1106e819bc17 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:48.947682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:48.947682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:48.996592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:48.998174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:48.998461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:48.998802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] PATH_INFO: `/v3/users/3f909a7a8b6340a2b062043cd53c8c84` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.000161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f909a7a8b6340a2b062043cd53c8c84)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.002134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.022258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04aaff-52d7-4e4d-92b8-b34ab8407a61 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.027174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.027936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.028220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.028494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.029178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.031197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.103098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:49.139164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1022/2029] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:48 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:49.139164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.139164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:49.205819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a7fd-208c-4cac-aee8-70850fc584cd admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.210826 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.211741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.212027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.212305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] PATH_INFO: `/v3/users/f450100f92cc4fc68388ae89224cd467` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.213526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f450100f92cc4fc68388ae89224cd467)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.215543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.231522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f909a7a8b6340a2b062043cd53c8c84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.239274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f909a7a8b6340a2b062043cd53c8c84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.240019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f909a7a8b6340a2b062043cd53c8c84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.251544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f909a7a8b6340a2b062043cd53c8c84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.314295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f909a7a8b6340a2b062043cd53c8c84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.321289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1008/2030] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:48 2022] DELETE /identity/v3/users/3f909a7a8b6340a2b062043cd53c8c84 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:49.322557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.323751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:49.352246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f450100f92cc4fc68388ae89224cd467'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.353765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f450100f92cc4fc68388ae89224cd467'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.366841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f450100f92cc4fc68388ae89224cd467'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.375759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f450100f92cc4fc68388ae89224cd467'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.390719 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0c61e-19c1-4496-a2d7-0a0845387685 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.395906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.396587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.396874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.397374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.397985 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.399947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.455274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-eca8739b-1147-4312-8286-262a05cf63c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f450100f92cc4fc68388ae89224cd467'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:49.462052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1023/2031] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:49 2022] DELETE /identity/v3/users/f450100f92cc4fc68388ae89224cd467 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:49.463490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.463777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.464052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.482063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:49.483611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:49.498532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:49.501557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1009/2032] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:49 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:49.507526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.509583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:49.524605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4df2a-a6fe-4cc2-8075-60864b306fc1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.553536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1024/2033] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:49.566352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.567834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:49.609640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.614799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.614799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.614799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.614799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.615844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.628646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1010/2034] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:49.632587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.645442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e34beb-953c-4835-816a-689bd4f49568 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.651471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.652597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.652956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.653245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] PATH_INFO: `/v3/projects/dcc5ffcff9164be1865a2950c61ab32d/users/774e9c39d47445b38070fd0106a804dd/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.671783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8342cd2-ffec-42d8-955e-d535bcd08917 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.673631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc5ffcff9164be1865a2950c61ab32d, user_id=774e9c39d47445b38070fd0106a804dd, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.675835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.676483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.676680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.676780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.677041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.688525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.689992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.700177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1025/2035] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:49.755630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1011/2036] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:49 2022] PUT /identity/v3/projects/dcc5ffcff9164be1865a2950c61ab32d/users/774e9c39d47445b38070fd0106a804dd/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:49.759501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.770620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a0b2-4c6d-427e-bb93-c8b4cdd54c5a service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.832019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.832695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.832981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.833263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.833998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.835640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.845711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1026/2037] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:49.849873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.859734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:49.862323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d778e68-f1a1-4fad-905d-9a3c28452321 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.864948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:49.876798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb19db-ffb5-4c96-b854-3acf5f3a80ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.877758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb19db-ffb5-4c96-b854-3acf5f3a80ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.878091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb19db-ffb5-4c96-b854-3acf5f3a80ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.878351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb19db-ffb5-4c96-b854-3acf5f3a80ee admin admin] PATH_INFO: `/v3/projects/dcc5ffcff9164be1865a2950c61ab32d/users/774e9c39d47445b38070fd0106a804dd/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.880395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb19db-ffb5-4c96-b854-3acf5f3a80ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc5ffcff9164be1865a2950c61ab32d, user_id=774e9c39d47445b38070fd0106a804dd, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.883094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb19db-ffb5-4c96-b854-3acf5f3a80ee admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.884591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a94e6-f37d-45a9-a39f-c93dfdc7db60 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:49.900084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:49.901113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.901610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.902045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] PATH_INFO: `/v3/projects/9aceace1f9a7496da5aff1c39a1a826c` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.904862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aceace1f9a7496da5aff1c39a1a826c)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:49.910351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:49.917058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1012/2038] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:49 2022] PUT /identity/v3/projects/dcc5ffcff9164be1865a2950c61ab32d/users/774e9c39d47445b38070fd0106a804dd/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:49.918770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:49.919209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:49.920364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:49.967762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] MFA Rules not processed for user `774e9c39d47445b38070fd0106a804dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:49.970324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.064122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aceace1f9a7496da5aff1c39a1a826c', 'request_id': 'req-a477da41-2ee9-4807-8d51-c939d0d7e32a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.067389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1013/2039] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:50.071173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aceace1f9a7496da5aff1c39a1a826c', 'request_id': 'req-a477da41-2ee9-4807-8d51-c939d0d7e32a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.082137 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.083835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.152127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34069951-e26c-491f-b397-89a5f9cca8ad None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.173041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:50.204085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aceace1f9a7496da5aff1c39a1a826c', 'request_id': 'req-a477da41-2ee9-4807-8d51-c939d0d7e32a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.209927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300957d-be0a-4a1e-bfd4-c8dec3a111aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.210596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300957d-be0a-4a1e-bfd4-c8dec3a111aa service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.210882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300957d-be0a-4a1e-bfd4-c8dec3a111aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.211161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300957d-be0a-4a1e-bfd4-c8dec3a111aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.220778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] The token cache is being invalidate because project 9aceace1f9a7496da5aff1c39a1a826c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:50.221463 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.222329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a477da41-2ee9-4807-8d51-c939d0d7e32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.224577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1027/2040] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:49 2022] DELETE /identity/v3/projects/9aceace1f9a7496da5aff1c39a1a826c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:50.229366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.229658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.229938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.249358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:50.251215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.281036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300957d-be0a-4a1e-bfd4-c8dec3a111aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.282540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300957d-be0a-4a1e-bfd4-c8dec3a111aa service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.292403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1014/2041] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:50.312298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1028/2042] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:50.324268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.324651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.325051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.335837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.339272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.355358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce686819-4085-47bf-a1b5-c6496b8f9c9a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.356510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:50.357990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.361342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.362217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.362587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.362924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] PATH_INFO: `/v3/projects/52bd1ed3d31141cfb9cd4eac2499194d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.364423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bd1ed3d31141cfb9cd4eac2499194d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.366607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.368437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1015/2043] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:50.371727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.381349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e75f9-e036-4d7c-a3ce-af46ef777823 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.386042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.386674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.386956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.387343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.387990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:50.388297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.389908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.401327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1016/2044] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:28:50.406327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.416480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407e993-c1e4-4bc7-a34b-77a5c6fb3269 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.421274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.421982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.422340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.422624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.423860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.425779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.480330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bd1ed3d31141cfb9cd4eac2499194d', 'request_id': 'req-7f83bf79-5560-476d-9b28-3b484493d034'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.491656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bd1ed3d31141cfb9cd4eac2499194d', 'request_id': 'req-7f83bf79-5560-476d-9b28-3b484493d034'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.526877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1017/2045] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:50 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:50.527897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.529486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.570847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bd1ed3d31141cfb9cd4eac2499194d', 'request_id': 'req-7f83bf79-5560-476d-9b28-3b484493d034'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.583552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] The token cache is being invalidate because project 52bd1ed3d31141cfb9cd4eac2499194d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:50.583931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.584654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.586278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1029/2046] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:50 2022] DELETE /identity/v3/projects/52bd1ed3d31141cfb9cd4eac2499194d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:50.586563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f96f6-1b74-48b7-8058-fae1457b0ed2 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.587216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.588529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.590611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.591286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.591606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.591889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.593736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.601923 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:50.655843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.657322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.660497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83bf79-5560-476d-9b28-3b484493d034 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.667017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1018/2047] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:50.667881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.669196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:50.726223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.726904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.727193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.727504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.728203 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.730157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.741446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d161c-7b3e-4157-969e-c23c7e29f24f service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.746094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.747071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.747382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.747663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] PATH_INFO: `/v3/users/3531399ff41a4b91a4852d608b65188c` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.748861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3531399ff41a4b91a4852d608b65188c)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.751385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.852684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1030/2048] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:50 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:50.859333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.870930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492045c0-dc5c-492c-9810-ef99adcfba5d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.877053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.877742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.878106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.878305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.879048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.880757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.894719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1031/2049] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:50.896347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.906769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8016a95-0731-4d13-b55a-c1438e76597a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.913010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.913398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.913682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.913968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.914699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.916352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:50.928690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:50.936081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1032/2050] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:50.940509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:50.949847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3531399ff41a4b91a4852d608b65188c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.957251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421af5e-d8d9-4d22-8618-62ada32ebad6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:50.959090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3531399ff41a4b91a4852d608b65188c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.959850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3531399ff41a4b91a4852d608b65188c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.964187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:50.967399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:50.967399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:50.967399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] PATH_INFO: `/v3/projects/607e5f7daaa24217ba9f830e05aa2015/users/f030c1a8e0a84648848e1327c3f7e230/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:50.974779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3531399ff41a4b91a4852d608b65188c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:50.982686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=607e5f7daaa24217ba9f830e05aa2015, user_id=f030c1a8e0a84648848e1327c3f7e230, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:50.985499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:51.045373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5ad93222-0e0c-42a3-8a9e-826a1fa229bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3531399ff41a4b91a4852d608b65188c'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.057061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1019/2051] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:50 2022] DELETE /identity/v3/users/3531399ff41a4b91a4852d608b65188c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:51.083104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1033/2052] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:50 2022] PUT /identity/v3/projects/607e5f7daaa24217ba9f830e05aa2015/users/f030c1a8e0a84648848e1327c3f7e230/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:51.093683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.094559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.094873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.159288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] MFA Rules not processed for user `f030c1a8e0a84648848e1327c3f7e230`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:51.161570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.250168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1020/2053] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:51.312346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:51.314277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.315939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:51.317408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.388800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f824b-690c-41e5-873e-a46d9eadcc33 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:51.391842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed09455-7be4-4f67-9be0-afdc72e8a4e1 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:51.395455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:51.396619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:51.397085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.397370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.397682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.397996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.398370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.398744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] PATH_INFO: `/v3/users/8d6dec4577db46fbaab5d0557345e1a9` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.400013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6dec4577db46fbaab5d0557345e1a9)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:51.402342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:51.411141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:51.414917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:51.420218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:51.427133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1021/2054] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:51.532911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:51.534962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.566765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffdfa2-1335-4c0b-a5a0-8ab2a1e230a3 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:51.571763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:51.572640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.572925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.573208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] PATH_INFO: `/v3/projects/ebd104c85e0242b2872c540ad62d5355` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.574502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd104c85e0242b2872c540ad62d5355)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:51.576463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:51.617122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6dec4577db46fbaab5d0557345e1a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.617886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6dec4577db46fbaab5d0557345e1a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.630513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6dec4577db46fbaab5d0557345e1a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.641436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6dec4577db46fbaab5d0557345e1a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.707241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd104c85e0242b2872c540ad62d5355', 'request_id': 'req-df05e701-f8d6-4a61-88d4-1f0605e75abd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.723934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b8264cc2-4a83-4b3a-a694-d99220499940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6dec4577db46fbaab5d0557345e1a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.731551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1034/2055] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:51 2022] DELETE /identity/v3/users/8d6dec4577db46fbaab5d0557345e1a9 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:51.736837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.737285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.737706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.756289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:51.757554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.819244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1035/2056] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:51.829090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:51.830631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.837979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd104c85e0242b2872c540ad62d5355', 'request_id': 'req-df05e701-f8d6-4a61-88d4-1f0605e75abd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.846041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd104c85e0242b2872c540ad62d5355', 'request_id': 'req-df05e701-f8d6-4a61-88d4-1f0605e75abd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.862122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] The token cache is being invalidate because project ebd104c85e0242b2872c540ad62d5355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:51.862574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.863511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-df05e701-f8d6-4a61-88d4-1f0605e75abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:51.869283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1022/2057] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:51 2022] DELETE /identity/v3/projects/ebd104c85e0242b2872c540ad62d5355 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:51.870698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212340a9-e08e-41ec-a1f0-f1f4e58cb3b3 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.870997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212340a9-e08e-41ec-a1f0-f1f4e58cb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.871285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212340a9-e08e-41ec-a1f0-f1f4e58cb3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.877158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:51.897069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-212340a9-e08e-41ec-a1f0-f1f4e58cb3b3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:51.898436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-212340a9-e08e-41ec-a1f0-f1f4e58cb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.908892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1023/2058] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:51.910248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f7e8-4665-4dfd-ab4d-b7209b24d715 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:51.916769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.917063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.917394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.918675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:51.919362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.919657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.919936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.921858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.936166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:51.937681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:51.941335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:51.942541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:51.947873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1036/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:51.953705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1024/2060] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:51.955334 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:51.965932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d3dfb-9222-4d16-b907-7af0215d0b08 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:51.971528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:51.972263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:51.972630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:51.972975 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:51.973827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:51.974140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:51.975940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:51.997552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1037/2061] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:28:52.002041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.011724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b07afd-3717-4909-87b9-ddc2765795a6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.017038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.017714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.018059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.018345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.019091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.025405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.146244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1025/2062] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:52.152366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.163097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83f3b8-6f87-4841-b17c-75539258ec0e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.248909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.249712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.249995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.250446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.251139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.253139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.360366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1038/2063] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:52 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:52.365153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.377024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727e344-699a-44f9-871d-db133917b0fb admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.382254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.382983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.383406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.383639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.384429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.386416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.398320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1026/2064] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:52.402110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.413214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e6f88-5667-4da2-b722-6b3dc08a0196 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.418742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.419453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.419778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.421544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.421544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.422481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.432898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1039/2065] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:52.437116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.448189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.449007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce7cf0-357f-46b7-95bf-49606702ce3a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.454411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730484d6-6d9c-4f50-8f9b-c9e347e207d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.461299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730484d6-6d9c-4f50-8f9b-c9e347e207d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.461521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:52.465623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730484d6-6d9c-4f50-8f9b-c9e347e207d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.465623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730484d6-6d9c-4f50-8f9b-c9e347e207d3 admin admin] PATH_INFO: `/v3/projects/a9a9fa498ad14cfea34dae1d6c94309d/users/80db9525e65e446aa758bf7d77935f71/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.473144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff74faf-ddb7-493a-95f9-f67077394674 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.480755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730484d6-6d9c-4f50-8f9b-c9e347e207d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a9fa498ad14cfea34dae1d6c94309d, user_id=80db9525e65e446aa758bf7d77935f71, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.483775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730484d6-6d9c-4f50-8f9b-c9e347e207d3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.486722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.487756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.487979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.488195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] PATH_INFO: `/v3/projects/e0d7f979a0854c3eac771832da27c37e` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.493258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0d7f979a0854c3eac771832da27c37e)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.493258 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.514285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:52.548247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1027/2066] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:52 2022] PUT /identity/v3/projects/a9a9fa498ad14cfea34dae1d6c94309d/users/80db9525e65e446aa758bf7d77935f71/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:52.558453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.558739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.559010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.585109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] MFA Rules not processed for user `80db9525e65e446aa758bf7d77935f71`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:52.586324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:52.621503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0d7f979a0854c3eac771832da27c37e', 'request_id': 'req-21d18b6b-94af-4ad2-b370-14ce4c48122d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:52.627812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0d7f979a0854c3eac771832da27c37e', 'request_id': 'req-21d18b6b-94af-4ad2-b370-14ce4c48122d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:52.661227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1028/2067] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:52.665532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.701647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0d7f979a0854c3eac771832da27c37e', 'request_id': 'req-21d18b6b-94af-4ad2-b370-14ce4c48122d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:52.717059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] The token cache is being invalidate because project e0d7f979a0854c3eac771832da27c37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:52.717436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:52.718232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:52.720690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1040/2068] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:52 2022] DELETE /identity/v3/projects/e0d7f979a0854c3eac771832da27c37e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:52.725400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.726992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:52.727745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9106e4-74e9-4066-b3a8-596cf2a859d7 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.757114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:52.798771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.799501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.799787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.800065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.800753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.803702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.805212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d18b6b-94af-4ad2-b370-14ce4c48122d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.810409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.811606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.812292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.813553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] PATH_INFO: `/v3/users/774e9c39d47445b38070fd0106a804dd` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.814364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774e9c39d47445b38070fd0106a804dd)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.816389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.891900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1029/2069] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:52.895764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:52.897377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:52.959355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a03e20-6591-48b8-93c7-a1ec0981d96d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:52.964050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:52.964748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:52.965040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:52.965324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:52.966030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:52.968008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:52.971232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:53.007898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774e9c39d47445b38070fd0106a804dd'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.008579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774e9c39d47445b38070fd0106a804dd'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.020941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774e9c39d47445b38070fd0106a804dd'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.027961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774e9c39d47445b38070fd0106a804dd'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.140576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774e9c39d47445b38070fd0106a804dd'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.147802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1030/2070] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:52 2022] POST /identity/v3/users => generated 300 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:53.148138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1041/2071] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:52 2022] DELETE /identity/v3/users/774e9c39d47445b38070fd0106a804dd => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:53.154222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.165426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:53.197912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d062f-bb36-4cbc-990a-4237508b2ea9 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.263769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.264183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.265166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.265166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.265531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.267805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.278309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1031/2072] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:53.285919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.296169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367c228-aeab-4f02-9ffe-74955f00b7e0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.301627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.302573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.302859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.303138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] PATH_INFO: `/v3/projects/c593b2261b954fad8d5f40ff8a633018/users/ad4a082039ab4f72a58fd9b2d8a17ea3/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.319503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c593b2261b954fad8d5f40ff8a633018, user_id=ad4a082039ab4f72a58fd9b2d8a17ea3, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.322167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.358496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:53.417448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1042/2073] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:53 2022] PUT /identity/v3/projects/c593b2261b954fad8d5f40ff8a633018/users/ad4a082039ab4f72a58fd9b2d8a17ea3/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:53.422704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.434648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2d156-b8fa-46e0-9817-37358185b3f0 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.509831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6a2b5-3e92-4e8b-8011-cad686a86d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.510612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a2b5-3e92-4e8b-8011-cad686a86d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.510961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a2b5-3e92-4e8b-8011-cad686a86d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.511319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a2b5-3e92-4e8b-8011-cad686a86d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.512294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6a2b5-3e92-4e8b-8011-cad686a86d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.513948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6a2b5-3e92-4e8b-8011-cad686a86d27 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.524350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1032/2074] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:28:53.528582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.539806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5468a2-4ef6-4b48-ac8b-d97edb75564c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.546838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.546838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.546838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.546838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] PATH_INFO: `/v3/projects/c593b2261b954fad8d5f40ff8a633018/users/ad4a082039ab4f72a58fd9b2d8a17ea3/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.548681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c593b2261b954fad8d5f40ff8a633018, user_id=ad4a082039ab4f72a58fd9b2d8a17ea3, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.551346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.562055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1043/2075] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:53 2022] PUT /identity/v3/projects/c593b2261b954fad8d5f40ff8a633018/users/ad4a082039ab4f72a58fd9b2d8a17ea3/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:53.573061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.573566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.575115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.601804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.603660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:53.604671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] MFA Rules not processed for user `ad4a082039ab4f72a58fd9b2d8a17ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:53.606107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:53.686833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1033/2076] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:53.698480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.699964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:53.713417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16d860-3f0b-4514-bacd-c4cef6daaf19 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.718427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.719393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.719677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.719956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] PATH_INFO: `/v3/projects/dcc5ffcff9164be1865a2950c61ab32d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.721243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc5ffcff9164be1865a2950c61ab32d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.723202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.753793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:53.761748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea6bc5-b01d-49c9-a1a9-fd1ec22ffa6f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.765609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.766936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.767215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.767505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.776423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.777947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.787262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1034/2077] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:53.811829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc5ffcff9164be1865a2950c61ab32d', 'request_id': 'req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.815911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.818434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc5ffcff9164be1865a2950c61ab32d', 'request_id': 'req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.826859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9755027-5d76-4030-b0e5-7946ff961b48 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:53.883828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:53.884712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.884994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.885267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] PATH_INFO: `/v3/users/80db9525e65e446aa758bf7d77935f71` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.886464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=80db9525e65e446aa758bf7d77935f71)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:53.888509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:53.889567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc5ffcff9164be1865a2950c61ab32d', 'request_id': 'req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.911252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] The token cache is being invalidate because project dcc5ffcff9164be1865a2950c61ab32d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:53.911587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.912393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b2d8eca8-7c77-4eea-b445-fc7de1fb87b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:53.913989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1044/2078] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:53 2022] DELETE /identity/v3/projects/dcc5ffcff9164be1865a2950c61ab32d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:53.918653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7598-2bd4-4fa9-84e1-c8141ac857a4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.918946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7598-2bd4-4fa9-84e1-c8141ac857a4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.919231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7598-2bd4-4fa9-84e1-c8141ac857a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.937427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d6db7598-2bd4-4fa9-84e1-c8141ac857a4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:53.939294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d6db7598-2bd4-4fa9-84e1-c8141ac857a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:53.947083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1045/2079] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:53.955535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:53.955827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:53.956102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:53.973701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:53.974866 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:53.983636 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1046/2080] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:53.988598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:53.997499 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb2afa-8d96-4473-8d3d-1b47bb8fd6ec None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.002784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.003459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.003742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.004067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.004730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:54.005034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.006639 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.018243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1047/2081] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:28:54.027497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.036779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dc671-1e0a-42c2-b99b-ae9bb774b40a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.041353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.041966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.042249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.042524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.047677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.047677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.079225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80db9525e65e446aa758bf7d77935f71'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:54.086015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80db9525e65e446aa758bf7d77935f71'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:54.086674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80db9525e65e446aa758bf7d77935f71'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:54.098175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80db9525e65e446aa758bf7d77935f71'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:54.186326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c7d72a38-6b30-4de7-94f2-ced19346b54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80db9525e65e446aa758bf7d77935f71'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:54.192943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1035/2082] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:53 2022] DELETE /identity/v3/users/80db9525e65e446aa758bf7d77935f71 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:54.197195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.197489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.197827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.217772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:54.219638 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:54.290736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1048/2083] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:54 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:28:54.290938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1036/2084] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:54.292303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.294063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:54.298450 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.299875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:54.343701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:54.359212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf7cfc-af64-4de0-9fe6-f5b8a69169d2 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.359505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8274625-d3d2-4c80-b218-f26bed2eaca5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.364648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.365331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.365619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.365906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.366602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.368821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.406098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:54.416015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.416708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.416996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.417279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.427958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.429444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.439677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1037/2085] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:54.460882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1049/2086] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:54 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:54.465145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.476901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9d4a0-2adf-4d5a-827e-057710eb13c0 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.482140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.483111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.483427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.483709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.484548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.486935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.502716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1038/2087] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:54.507177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.522439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d8848-b9b4-4d86-b4c7-7c2140233be0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.527696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b91fb5-f0e1-471f-8e48-02fd629b50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.528443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b91fb5-f0e1-471f-8e48-02fd629b50ad admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.528797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b91fb5-f0e1-471f-8e48-02fd629b50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.529197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b91fb5-f0e1-471f-8e48-02fd629b50ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.530443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b91fb5-f0e1-471f-8e48-02fd629b50ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.532195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b91fb5-f0e1-471f-8e48-02fd629b50ad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.550818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1050/2088] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:54.555150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.569381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6e646-2b76-43be-b660-14433f9e995e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.576312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.577585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.577945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.578157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] PATH_INFO: `/v3/projects/cccc6823690047a2a64e5ce7389de5fe/users/32ee84c10f0d4ac09e0e2cbfba0241d1/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.595452 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccc6823690047a2a64e5ce7389de5fe, user_id=32ee84c10f0d4ac09e0e2cbfba0241d1, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.598230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.695367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1039/2089] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:54 2022] PUT /identity/v3/projects/cccc6823690047a2a64e5ce7389de5fe/users/32ee84c10f0d4ac09e0e2cbfba0241d1/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:54.697170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.697536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.698448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.707604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:54.755335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] MFA Rules not processed for user `32ee84c10f0d4ac09e0e2cbfba0241d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:54.756628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:54.841271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1051/2090] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:54.853175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.855066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:54.916380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd00c6-30d1-4c4e-a220-a146d7664c04 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:54.920171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:54.920830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:54.921120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:54.921404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:54.931006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:54.932351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:54.942203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1040/2091] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:28:54.960019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:54.962699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:54.964548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:54.967847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:55.034647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1557b9-5cf7-4618-b9b3-71fbd28fe515 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:55.040808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:55.041673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d679c-1228-4e90-bc8d-3309dc91f3ac None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:55.041781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:55.042147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:55.042510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] PATH_INFO: `/v3/users/32ee84c10f0d4ac09e0e2cbfba0241d1` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:55.043784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ee84c10f0d4ac09e0e2cbfba0241d1)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:55.045723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:55.046699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:55.048253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:55.048548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:55.048828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] PATH_INFO: `/v3/projects/a9a9fa498ad14cfea34dae1d6c94309d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:55.050887 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a9fa498ad14cfea34dae1d6c94309d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:55.052899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:55.064141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:55.307564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ee84c10f0d4ac09e0e2cbfba0241d1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.314410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ee84c10f0d4ac09e0e2cbfba0241d1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.315107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ee84c10f0d4ac09e0e2cbfba0241d1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.317266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a9fa498ad14cfea34dae1d6c94309d', 'request_id': 'req-836f6ae5-587f-4c48-8490-31e9215e22de'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.322797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a9fa498ad14cfea34dae1d6c94309d', 'request_id': 'req-836f6ae5-587f-4c48-8490-31e9215e22de'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.327665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ee84c10f0d4ac09e0e2cbfba0241d1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.353673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:55.422286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a9fa498ad14cfea34dae1d6c94309d', 'request_id': 'req-836f6ae5-587f-4c48-8490-31e9215e22de'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.434444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] The token cache is being invalidate because project a9a9fa498ad14cfea34dae1d6c94309d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:55.434742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.435474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.437223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1052/2092] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:54 2022] DELETE /identity/v3/projects/a9a9fa498ad14cfea34dae1d6c94309d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:55.440161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:55.441655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:55.443221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:55.485412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-01313546-d7ca-4483-b60b-34310ded6772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ee84c10f0d4ac09e0e2cbfba0241d1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.493759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1041/2093] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:54 2022] DELETE /identity/v3/users/32ee84c10f0d4ac09e0e2cbfba0241d1 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:55.498739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:55.499153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:55.499642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:55.512649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f6ae5-587f-4c48-8490-31e9215e22de admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:55.520363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:55.521723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:55.582759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:55.584048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:55.584048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:55.584244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] PATH_INFO: `/v3/users/ad4a082039ab4f72a58fd9b2d8a17ea3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:55.585414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4a082039ab4f72a58fd9b2d8a17ea3)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:55.587690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:55.593277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1042/2094] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:55.600819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:55.602827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:55.671264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d44ac-1c43-4f21-8923-1fa64f485be8 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:55.675651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:55.676466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:55.676818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:55.677162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:55.697778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:55.699406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:55.711539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1043/2095] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:55.778231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4a082039ab4f72a58fd9b2d8a17ea3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.779381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4a082039ab4f72a58fd9b2d8a17ea3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.794912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4a082039ab4f72a58fd9b2d8a17ea3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.810226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4a082039ab4f72a58fd9b2d8a17ea3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.902870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:55.937275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4a082039ab4f72a58fd9b2d8a17ea3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:55.946609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1053/2096] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:28:55 2022] DELETE /identity/v3/users/ad4a082039ab4f72a58fd9b2d8a17ea3 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:55.967390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:56.090149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:56.091930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:56.183546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052940ac-6b52-4a3a-8112-4225a7b17fea service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:56.243054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:56.243788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:56.244075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:56.244356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:56.246197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:56.312304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:56.316952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:56.317046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:56.318945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:56.335860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1044/2097] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:56.408045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd87f1-4c39-4ef5-a295-88ef736ef14b admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:56.413882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:56.414994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:56.415402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:56.415754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] PATH_INFO: `/v3/projects/cccc6823690047a2a64e5ce7389de5fe` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:56.417074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccc6823690047a2a64e5ce7389de5fe)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:56.419785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:56.569472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccc6823690047a2a64e5ce7389de5fe', 'request_id': 'req-482312f7-e623-4efe-abaa-80eeea3200d7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.574779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccc6823690047a2a64e5ce7389de5fe', 'request_id': 'req-482312f7-e623-4efe-abaa-80eeea3200d7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.586735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:56.589135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:56.605964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de9efc-bdeb-4f2c-9ea7-b233d505cf52 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:56.668536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccc6823690047a2a64e5ce7389de5fe', 'request_id': 'req-482312f7-e623-4efe-abaa-80eeea3200d7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.679211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:56.680395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:56.680756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:56.681138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] PATH_INFO: `/v3/projects/c593b2261b954fad8d5f40ff8a633018` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:56.682519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c593b2261b954fad8d5f40ff8a633018)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:56.683554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] The token cache is being invalidate because project cccc6823690047a2a64e5ce7389de5fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:56.683856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.684871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:56.685586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-482312f7-e623-4efe-abaa-80eeea3200d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.688835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1054/2098] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:56 2022] DELETE /identity/v3/projects/cccc6823690047a2a64e5ce7389de5fe => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:56.818116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c593b2261b954fad8d5f40ff8a633018', 'request_id': 'req-ebc0a1b8-8e0f-4306-b598-5a7715c402de'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.901416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:56.904440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c593b2261b954fad8d5f40ff8a633018', 'request_id': 'req-ebc0a1b8-8e0f-4306-b598-5a7715c402de'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.909173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c593b2261b954fad8d5f40ff8a633018', 'request_id': 'req-ebc0a1b8-8e0f-4306-b598-5a7715c402de'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.929455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] The token cache is being invalidate because project c593b2261b954fad8d5f40ff8a633018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:28:56.929761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.930511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ebc0a1b8-8e0f-4306-b598-5a7715c402de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:28:56.935991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1045/2099] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:28:56 2022] DELETE /identity/v3/projects/c593b2261b954fad8d5f40ff8a633018 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:56.938446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:56.938947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:56.939027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:56.968241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:56.969550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:57.054569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1055/2100] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:57.064718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.065722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.075624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.171044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:57.172306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:57.182073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1046/2101] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:57.186595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.197314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c0eb5-be17-412a-87cc-e9611ed45f80 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.202479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.203138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.203452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.203731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.204456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:28:57.204762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.206424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:57.220265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1056/2102] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:28:57.225270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.235096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c2ab5-83f8-45fc-97f4-d68a6b85a7ab None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.241039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.241861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.242185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.242470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.243175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.245149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:57.350326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1047/2103] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:57 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:57.355532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.367123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3952-4161-4636-9524-adf1af796dcc admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.420852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:57.441765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.444564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.444564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.444564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.444564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.445780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:57.508125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1057/2104] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:57 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:57.512174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.522981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63203cb0-f89a-4ebc-8c17-e7023cf9711a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.527753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.528424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.528709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.528988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.529723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.531387 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:57.544669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1048/2105] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:57.548968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.558892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5eec6-5a3d-456d-b395-a97a50c885d1 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.563718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc84969-d313-4d69-8382-543519b54010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.564368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc84969-d313-4d69-8382-543519b54010 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.564650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc84969-d313-4d69-8382-543519b54010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.564926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc84969-d313-4d69-8382-543519b54010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.565697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc84969-d313-4d69-8382-543519b54010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.567321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc84969-d313-4d69-8382-543519b54010 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:57.577692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1058/2106] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:57.582519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.592690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22f32-623c-4fd2-8638-7119e1caa214 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.597986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.598864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.599146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.599455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] PATH_INFO: `/v3/projects/05cd537503f0438a8fb38b9118d7825b/users/fdaece153c9f4762ab74c1ddb0f91684/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.616374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05cd537503f0438a8fb38b9118d7825b, user_id=fdaece153c9f4762ab74c1ddb0f91684, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.617004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:57.709098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1049/2107] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:57 2022] PUT /identity/v3/projects/05cd537503f0438a8fb38b9118d7825b/users/fdaece153c9f4762ab74c1ddb0f91684/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:57.719459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.719785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.720072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.752896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] MFA Rules not processed for user `fdaece153c9f4762ab74c1ddb0f91684`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:57.754252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:57.807016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:57.834992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1059/2108] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:28:57.839892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:57.912508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b4c9d-2743-4f08-9e13-2c82bce30b27 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:57.917171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:57.917956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:57.918173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:57.918454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:57.919167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:57.921121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:58.019104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1050/2109] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:28:57 2022] POST /identity/v3/projects => generated 393 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:58.024384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:58.039855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efd460-f416-4ea8-8ca6-c060d89c016b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:58.103042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:58.103776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.104043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.104324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.105078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:58.107147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:58.241774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1060/2110] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:28:58 2022] POST /identity/v3/users => generated 303 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:28:58.245501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:58.259898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb2dc1-6c35-4c43-a444-516fc5fd5fbe admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:58.264591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:58.265274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.265559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.265839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.266713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:58.268907 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:58.280848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1051/2111] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:58.286129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:58.294713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:58.297804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a9a5e-4abb-49b1-8187-29630757f20f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:58.303371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:58.304277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.304572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.304857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] PATH_INFO: `/v3/projects/721ce5fcc3284f7090bfcdf35d91eb00/users/84068790c7f6460ca84f6ec81a8613c9/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.322118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] RBAC: Authorizing `identity:create_grant(project_id=721ce5fcc3284f7090bfcdf35d91eb00, user_id=84068790c7f6460ca84f6ec81a8613c9, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:58.324890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:58.590815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1061/2112] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:58 2022] PUT /identity/v3/projects/721ce5fcc3284f7090bfcdf35d91eb00/users/84068790c7f6460ca84f6ec81a8613c9/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:28:58.622119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:58.634371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ec996-9460-401c-9854-b2e1f9d8e3c8 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:58.751357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c74c-a7cb-4432-a776-26af728fca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:58.751357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c74c-a7cb-4432-a776-26af728fca2f admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.751357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c74c-a7cb-4432-a776-26af728fca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.751357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c74c-a7cb-4432-a776-26af728fca2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.752237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c74c-a7cb-4432-a776-26af728fca2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:58.752237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c74c-a7cb-4432-a776-26af728fca2f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:58.752237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1052/2113] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:28:58.753173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:58.758653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecad63-ec26-42c3-89ba-22de3040868d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:58.764874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:58.765837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.766118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.766454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] PATH_INFO: `/v3/projects/721ce5fcc3284f7090bfcdf35d91eb00/users/84068790c7f6460ca84f6ec81a8613c9/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.769195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=721ce5fcc3284f7090bfcdf35d91eb00, user_id=84068790c7f6460ca84f6ec81a8613c9, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:58.772209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:58.783710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1062/2114] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:28:58 2022] PUT /identity/v3/projects/721ce5fcc3284f7090bfcdf35d91eb00/users/84068790c7f6460ca84f6ec81a8613c9/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:28:58.795765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.798744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.798744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.825937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] MFA Rules not processed for user `84068790c7f6460ca84f6ec81a8613c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:28:58.827973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:58.901877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1053/2115] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:28:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:28:58.913137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:58.915519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:58.978191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cc87e-994b-4357-8a18-f57867b779a0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:58.981983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:58.983016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:58.983319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:58.983601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:58.993407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:58.994847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:59.005255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1063/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:59.040392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:59.058831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:28:59.060499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:28:59.073804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d35d80-15c5-4f3c-9825-df562b49d06f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:28:59.078564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:28:59.079341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:28:59.079678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:28:59.079960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:28:59.141511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:28:59.142969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:28:59.157536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1054/2117] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:28:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:28:59.400380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:28:59.508555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:00.012229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:00.066909 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:00.295268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:00.323596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:00.632495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:06.175387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:06.176920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:06.248631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707f58d-0795-4a8f-a274-db13fb678c4a service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:06.253534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:06.254944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:06.255242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:06.255541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] PATH_INFO: `/v3/users/c2eb70d1582b4876931f05cd4015a346` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:06.256788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2eb70d1582b4876931f05cd4015a346)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:06.259080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:06.275645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:06.447701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2eb70d1582b4876931f05cd4015a346'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:06.448365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2eb70d1582b4876931f05cd4015a346'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:06.461247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2eb70d1582b4876931f05cd4015a346'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:06.467983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2eb70d1582b4876931f05cd4015a346'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:06.518954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-aee29dd5-e271-49cc-a282-dea18e565a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2eb70d1582b4876931f05cd4015a346'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:06.525678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1064/2118] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:06 2022] DELETE /identity/v3/users/c2eb70d1582b4876931f05cd4015a346 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:06.772348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:06.784647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374186-d225-4866-aad1-f28fad117d28 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:06.843106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:06.844007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:06.844300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:06.844582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] PATH_INFO: `/v3/projects/8eb15379902c4745a1e3fd5b83d0e4be` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:06.845761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb15379902c4745a1e3fd5b83d0e4be)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:06.847711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:06.974098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:06.977013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb15379902c4745a1e3fd5b83d0e4be', 'request_id': 'req-97fea8bf-3833-4969-93be-c1535f4ec4bf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:07.073540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb15379902c4745a1e3fd5b83d0e4be', 'request_id': 'req-97fea8bf-3833-4969-93be-c1535f4ec4bf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:07.079274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb15379902c4745a1e3fd5b83d0e4be', 'request_id': 'req-97fea8bf-3833-4969-93be-c1535f4ec4bf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:07.091287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] The token cache is being invalidate because project 8eb15379902c4745a1e3fd5b83d0e4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:07.091708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:07.092348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97fea8bf-3833-4969-93be-c1535f4ec4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:07.093873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1055/2119] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:06 2022] DELETE /identity/v3/projects/8eb15379902c4745a1e3fd5b83d0e4be => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:07.098945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.099236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.099547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.116987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:07.118227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:07.176915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1065/2120] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:07.184915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.185237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.185520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.204219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:07.205432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:07.213328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1056/2121] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:07.217427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.226859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289434d-5799-4b99-9a09-aff3ea292742 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.231487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.232136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.232691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.232973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.233720 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:07.234029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.235675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.247640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1066/2122] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:29:07.251756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.261537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e6da8-9512-49a2-893c-695b6d1ddf7f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.266156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.266800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.267087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.267402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.268094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.269982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.353917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1057/2123] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:07 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:07.358278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.368438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946e6d5-f7d7-44df-82e3-b4920aa1f333 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.424986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.425661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.426217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.426341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.426935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.430247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.512919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1067/2124] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:07 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:07.516763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.527607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86baf67-eb2e-4c28-b98d-a8a73550be68 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.532227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.532885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.536487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.536487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.536487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.536487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.547087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1058/2125] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:07.551051 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.561740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672eb377-5761-4835-804c-3e13f4e16235 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.566402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38be23-22dd-4978-93d7-c30b6cb834fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.567632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38be23-22dd-4978-93d7-c30b6cb834fb admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.567930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38be23-22dd-4978-93d7-c30b6cb834fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.568211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38be23-22dd-4978-93d7-c30b6cb834fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.569055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38be23-22dd-4978-93d7-c30b6cb834fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.570533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38be23-22dd-4978-93d7-c30b6cb834fb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.572965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:07.584798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1068/2126] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:29:07.588667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.599769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a0da5-b9fe-4a5d-b82b-dd61b324d856 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.604669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.605516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.605779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.606051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] PATH_INFO: `/v3/projects/89e9dc06c3304c42a7fb6cbed1e357a4/users/a0a147daff9846e699c26259fac4e004/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.631664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e9dc06c3304c42a7fb6cbed1e357a4, user_id=a0a147daff9846e699c26259fac4e004, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.635130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.731488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1059/2127] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:07 2022] PUT /identity/v3/projects/89e9dc06c3304c42a7fb6cbed1e357a4/users/a0a147daff9846e699c26259fac4e004/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:07.736691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.736981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.737256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.754862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:07.756112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:07.813980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1069/2128] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:07.823487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:07.825425 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:07.882119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f3484-2515-4ef2-870e-c161d553379d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:07.885668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558afc0-1518-42cf-a9a7-cb125238e4b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:07.886308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558afc0-1518-42cf-a9a7-cb125238e4b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:07.886594 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558afc0-1518-42cf-a9a7-cb125238e4b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:07.886872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558afc0-1518-42cf-a9a7-cb125238e4b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:07.895703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558afc0-1518-42cf-a9a7-cb125238e4b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:07.897127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558afc0-1518-42cf-a9a7-cb125238e4b6 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:07.906393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1060/2129] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:10.191819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:10.194242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:10.206998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a514e-7578-47fc-81e9-9db33e5156d9 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:10.212884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:10.214100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:10.214385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:10.214681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] PATH_INFO: `/v3/users/9b1f72dd7ab6468c9600ce8e4e2f5dc0` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:10.221303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b1f72dd7ab6468c9600ce8e4e2f5dc0)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:10.221303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:10.461568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b1f72dd7ab6468c9600ce8e4e2f5dc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:10.461738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b1f72dd7ab6468c9600ce8e4e2f5dc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:10.479769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b1f72dd7ab6468c9600ce8e4e2f5dc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:10.487749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b1f72dd7ab6468c9600ce8e4e2f5dc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:10.563407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b1f72dd7ab6468c9600ce8e4e2f5dc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:10.571581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1070/2130] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:10 2022] DELETE /identity/v3/users/9b1f72dd7ab6468c9600ce8e4e2f5dc0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:10.576082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:10.576579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:10.576671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:10.594913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:10.596169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:10.669757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1061/2131] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:10.679881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:10.681319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:10.748267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd76836-e101-4e24-956d-76f56f642d6f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:10.752728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:10.753259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:10.753663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:10.753947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:10.764278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:10.765753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:10.776016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1071/2132] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:11.405265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:11.407064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:11.424370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29c00e-687a-4f7a-b2f8-c4a6bdf0d77f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:11.427122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:11.428157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:11.428580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:11.428968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] PATH_INFO: `/v3/projects/3284a94a4c3b4f2a96b8e266902d14e9` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:11.430327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3284a94a4c3b4f2a96b8e266902d14e9)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:11.432478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:11.450347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:11.621483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3284a94a4c3b4f2a96b8e266902d14e9', 'request_id': 'req-37be2092-a76b-4875-a8f0-8445159692fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:11.649317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:11.651603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:11.674966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a8dc-0b23-491c-8a26-8db86144aef5 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:11.686737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:11.731369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3284a94a4c3b4f2a96b8e266902d14e9', 'request_id': 'req-37be2092-a76b-4875-a8f0-8445159692fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:11.740334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3284a94a4c3b4f2a96b8e266902d14e9', 'request_id': 'req-37be2092-a76b-4875-a8f0-8445159692fc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:11.764614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] The token cache is being invalidate because project 3284a94a4c3b4f2a96b8e266902d14e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:11.765091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:11.769245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:11.771154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1062/2133] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:11 2022] DELETE /identity/v3/projects/3284a94a4c3b4f2a96b8e266902d14e9 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:11.775464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:11.777591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:11.788690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:11.793490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:11.793803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:11.794077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] PATH_INFO: `/v3/users/fdaece153c9f4762ab74c1ddb0f91684` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:11.796412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdaece153c9f4762ab74c1ddb0f91684)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:11.800176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:11.804926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2092-a76b-4875-a8f0-8445159692fc admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:11.811515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:11.812743 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:11.813640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:11.813640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] PATH_INFO: `/v3/users/8eab6df66a0649eaa0c5f661fd7b18dd` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:11.815141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eab6df66a0649eaa0c5f661fd7b18dd)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:11.817851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:12.049674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eab6df66a0649eaa0c5f661fd7b18dd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.057342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdaece153c9f4762ab74c1ddb0f91684'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.058990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdaece153c9f4762ab74c1ddb0f91684'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.064195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eab6df66a0649eaa0c5f661fd7b18dd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.065299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eab6df66a0649eaa0c5f661fd7b18dd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.084574 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdaece153c9f4762ab74c1ddb0f91684'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.087414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eab6df66a0649eaa0c5f661fd7b18dd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.095790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdaece153c9f4762ab74c1ddb0f91684'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.180616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eab6df66a0649eaa0c5f661fd7b18dd'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.188054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1063/2134] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:11 2022] DELETE /identity/v3/users/8eab6df66a0649eaa0c5f661fd7b18dd => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:12.191243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:12.191243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:12.240765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bca56a7c-8c69-4e17-a086-67dfa51e2e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdaece153c9f4762ab74c1ddb0f91684'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.250401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1072/2135] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:11 2022] DELETE /identity/v3/users/fdaece153c9f4762ab74c1ddb0f91684 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:12.257562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:12.258290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:12.258784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:12.311470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:12.312819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:12.336054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf3f81-626b-4c07-a54e-1cebe3552b1e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:12.427654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1073/2136] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:12.447640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:12.456957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:12.470834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:12.472538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:12.473020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:12.473707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] PATH_INFO: `/v3/users/4d3f819f1a8d4db99774cef9d9a0b493` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:12.475915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:12.477546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3f819f1a8d4db99774cef9d9a0b493)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:12.485408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:12.538415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33be9dc-9d32-4b74-820f-a0b4e5690d11 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:12.547843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb60de-2666-4cb6-8f05-3dc56a00a3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:12.548530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb60de-2666-4cb6-8f05-3dc56a00a3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:12.548814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb60de-2666-4cb6-8f05-3dc56a00a3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:12.549091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb60de-2666-4cb6-8f05-3dc56a00a3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:12.565254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb60de-2666-4cb6-8f05-3dc56a00a3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:12.568974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb60de-2666-4cb6-8f05-3dc56a00a3b9 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:12.586591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1074/2137] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:12.729502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3f819f1a8d4db99774cef9d9a0b493'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.738610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3f819f1a8d4db99774cef9d9a0b493'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.740247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3f819f1a8d4db99774cef9d9a0b493'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.749297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:12.756656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3f819f1a8d4db99774cef9d9a0b493'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.848076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3f819f1a8d4db99774cef9d9a0b493'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:12.865830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1064/2138] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:12 2022] DELETE /identity/v3/users/4d3f819f1a8d4db99774cef9d9a0b493 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:12.867480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:12.867905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:12.868421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:12.890663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:12.892314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:12.970040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1075/2139] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:12.981039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:12.983146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:13.051774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:13.053701 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:13.077774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9aeede-980e-4fd9-a851-fd87dbf63143 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:13.089079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:13.089969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.090256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.090538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] PATH_INFO: `/v3/projects/a6ac7833edb94f5c8f9411ebc95064aa` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.094099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ac7833edb94f5c8f9411ebc95064aa)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:13.099566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:13.126702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292cb78-97aa-44e6-bc79-c15967106b67 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:13.155447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:13.158953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.159153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.160418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.191626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:13.195373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:13.220703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1065/2140] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:13.375527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ac7833edb94f5c8f9411ebc95064aa', 'request_id': 'req-46c30b84-58f4-472f-9835-d0291dc7da6e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:13.384006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ac7833edb94f5c8f9411ebc95064aa', 'request_id': 'req-46c30b84-58f4-472f-9835-d0291dc7da6e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:13.511372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ac7833edb94f5c8f9411ebc95064aa', 'request_id': 'req-46c30b84-58f4-472f-9835-d0291dc7da6e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:13.523175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:13.526777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:13.533425 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] The token cache is being invalidate because project a6ac7833edb94f5c8f9411ebc95064aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:13.533748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:13.534399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-46c30b84-58f4-472f-9835-d0291dc7da6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:13.538901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1076/2141] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:13 2022] DELETE /identity/v3/projects/a6ac7833edb94f5c8f9411ebc95064aa => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:13.544569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98fab9-681a-4b8e-927d-1da71f221b89 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.544860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98fab9-681a-4b8e-927d-1da71f221b89 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.547586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98fab9-681a-4b8e-927d-1da71f221b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.558349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccd087-c6dc-476f-9b6b-f9688f6d2212 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:13.572608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-7c98fab9-681a-4b8e-927d-1da71f221b89 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:13.573352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7c98fab9-681a-4b8e-927d-1da71f221b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:13.775394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:13.778100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.778626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.779218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] PATH_INFO: `/v3/projects/05cd537503f0438a8fb38b9118d7825b` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.780994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05cd537503f0438a8fb38b9118d7825b)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:13.782148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1077/2142] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:13.784705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:13.790285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.790971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.791435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.819922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:13.821588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:13.834546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1078/2143] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:13.840530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:13.849654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5046d3-cf78-4c22-9630-bfc4b07ccaca None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:13.856850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:13.857805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.858095 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.858419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.859170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:13.859559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:13.861309 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:13.882550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1079/2144] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:29:13.885434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:13.897492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:13.915573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80363a0f-e475-4161-9da8-d1cd59cb55a4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:13.920694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:13.921364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:13.922170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:13.922461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:13.923959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:13.929545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:13.954621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05cd537503f0438a8fb38b9118d7825b', 'request_id': 'req-3f68dc66-84ba-4a00-9360-7b1d28cb9344'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.056936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05cd537503f0438a8fb38b9118d7825b', 'request_id': 'req-3f68dc66-84ba-4a00-9360-7b1d28cb9344'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.064019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05cd537503f0438a8fb38b9118d7825b', 'request_id': 'req-3f68dc66-84ba-4a00-9360-7b1d28cb9344'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.073686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1080/2145] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:13 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:14.073686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.073686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.080989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] The token cache is being invalidate because project 05cd537503f0438a8fb38b9118d7825b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:14.081601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.083356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.088840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1066/2146] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:13 2022] DELETE /identity/v3/projects/05cd537503f0438a8fb38b9118d7825b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:14.089423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.091767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.174476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa472d-220f-4ed6-9db6-f6db7e563b38 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.184133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.187025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.187331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.187871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] PATH_INFO: `/v3/projects/e40a39858afa41bb96f902c6178d44f3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.189554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40a39858afa41bb96f902c6178d44f3)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.191855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:14.206639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68dc66-84ba-4a00-9360-7b1d28cb9344 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.212672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.213494 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.214030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.214523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.215493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.217674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:14.339112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1067/2147] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:14 2022] POST /identity/v3/users => generated 282 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:14.340246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.341990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.360070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bc1eb-17d2-49b2-b061-e35e7dca4c71 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.367206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:14.369633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.370847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.371416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.371626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] PATH_INFO: `/v3/users/84068790c7f6460ca84f6ec81a8613c9` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.374705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84068790c7f6460ca84f6ec81a8613c9)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.375203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:14.396664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40a39858afa41bb96f902c6178d44f3', 'request_id': 'req-e677c713-101a-4e90-a729-eb55447d5d1e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.407884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40a39858afa41bb96f902c6178d44f3', 'request_id': 'req-e677c713-101a-4e90-a729-eb55447d5d1e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.512676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40a39858afa41bb96f902c6178d44f3', 'request_id': 'req-e677c713-101a-4e90-a729-eb55447d5d1e'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.527117 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] The token cache is being invalidate because project e40a39858afa41bb96f902c6178d44f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:14.527480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.528174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.530459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1081/2148] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:14 2022] DELETE /identity/v3/projects/e40a39858afa41bb96f902c6178d44f3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:14.534771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.536431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.606195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84068790c7f6460ca84f6ec81a8613c9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.616036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c713-101a-4e90-a729-eb55447d5d1e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.616509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84068790c7f6460ca84f6ec81a8613c9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.621685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84068790c7f6460ca84f6ec81a8613c9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.626343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d1d6f-926d-46bf-bc0a-1b22ec9dd6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.627077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d1d6f-926d-46bf-bc0a-1b22ec9dd6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.627619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d1d6f-926d-46bf-bc0a-1b22ec9dd6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.627920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d1d6f-926d-46bf-bc0a-1b22ec9dd6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.628750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d1d6f-926d-46bf-bc0a-1b22ec9dd6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.630970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d1d6f-926d-46bf-bc0a-1b22ec9dd6b8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:14.633396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84068790c7f6460ca84f6ec81a8613c9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.642234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1082/2149] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:14.642234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.642234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.642234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.671977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] MFA Rules not processed for user `a0a147daff9846e699c26259fac4e004`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:14.673168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.719690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84068790c7f6460ca84f6ec81a8613c9'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:14.728529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1068/2150] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:14 2022] DELETE /identity/v3/users/84068790c7f6460ca84f6ec81a8613c9 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:14.729676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.731216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.740321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1083/2151] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:14.741266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.742634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.761771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953142a-5c36-4bcf-b7f3-39cc04e69fea None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.823621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87eec0-7d48-4620-ad53-90974a0fea91 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.828375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.828964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.829867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.830305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.830609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.830898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.831007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.831382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] PATH_INFO: `/v3/users/f4f1da3d6537498a8c65f368a6c6dc5d` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.832209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.834929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4f1da3d6537498a8c65f368a6c6dc5d)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.835077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:14.837305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:14.846490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1084/2152] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:14.847180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:14.849683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:14.922559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb546448-48d4-427c-b5c3-77e8dcae06e3 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:14.928929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:14.929566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:14.929850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:14.930127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:14.989553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:14.991441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.001784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1085/2153] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:15.002657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.015033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2658b85-c537-4e4e-82fa-505500df7736 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.020787 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6f0a9-e79b-4adf-8d20-94967aa6c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.025192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6f0a9-e79b-4adf-8d20-94967aa6c27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.025544 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6f0a9-e79b-4adf-8d20-94967aa6c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.025969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6f0a9-e79b-4adf-8d20-94967aa6c27f admin admin] PATH_INFO: `/v3/projects/09cba523ef8e459db613b4f4ed5d4093/users/87b7124a042e476992a1b9ba82cf19f3/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.062213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6f0a9-e79b-4adf-8d20-94967aa6c27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cba523ef8e459db613b4f4ed5d4093, user_id=87b7124a042e476992a1b9ba82cf19f3, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.069508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6f0a9-e79b-4adf-8d20-94967aa6c27f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.089781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4f1da3d6537498a8c65f368a6c6dc5d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.098800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4f1da3d6537498a8c65f368a6c6dc5d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.099775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4f1da3d6537498a8c65f368a6c6dc5d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.118191 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4f1da3d6537498a8c65f368a6c6dc5d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.181766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4f1da3d6537498a8c65f368a6c6dc5d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.189545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1086/2154] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:15 2022] PUT /identity/v3/projects/09cba523ef8e459db613b4f4ed5d4093/users/87b7124a042e476992a1b9ba82cf19f3/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:15.203117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1069/2155] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:14 2022] DELETE /identity/v3/users/f4f1da3d6537498a8c65f368a6c6dc5d => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:15.203697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.204150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.204735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.237660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.240216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] MFA Rules not processed for user `87b7124a042e476992a1b9ba82cf19f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:15.240856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.241430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.353364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1087/2156] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:15.358876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.361584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.386986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd62f5-00b5-4422-9b83-079ec7390675 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.394028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.394614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.398431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.399178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] PATH_INFO: `/v3/projects/721ce5fcc3284f7090bfcdf35d91eb00` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.400776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721ce5fcc3284f7090bfcdf35d91eb00)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.403051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.499534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457abb7-3b8e-41a4-ae30-ab9b8b8ddda3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.505312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.506023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.506366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.506964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.507825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.509932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.609041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721ce5fcc3284f7090bfcdf35d91eb00', 'request_id': 'req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.656791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1088/2157] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:15 2022] POST /identity/v3/projects => generated 363 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:15.661458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.737361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fc3d9-ed0e-4199-b481-89ae702d2b44 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.742976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721ce5fcc3284f7090bfcdf35d91eb00', 'request_id': 'req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.744676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.745621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.745905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.746181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.747231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.760324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.760437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721ce5fcc3284f7090bfcdf35d91eb00', 'request_id': 'req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.773571 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] The token cache is being invalidate because project 721ce5fcc3284f7090bfcdf35d91eb00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:15.774036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.774897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d9b7581a-0b6f-43cd-a96f-fecf5898ef47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:15.778021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1070/2158] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:15 2022] DELETE /identity/v3/projects/721ce5fcc3284f7090bfcdf35d91eb00 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:15.788698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0055d52-13c8-48c7-8e67-8ac0cecd00e9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.789212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0055d52-13c8-48c7-8e67-8ac0cecd00e9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.789640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0055d52-13c8-48c7-8e67-8ac0cecd00e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.807885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d0055d52-13c8-48c7-8e67-8ac0cecd00e9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:15.809114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d0055d52-13c8-48c7-8e67-8ac0cecd00e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.819224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1071/2159] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:15.827223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.828217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.828568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.839887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:15.851246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:15.852596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.862033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1072/2160] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:15.877784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1089/2161] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:15 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:15.882850 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.885759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.893434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.903899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126b7e8-209a-4da0-94d3-49a5a845ec1a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.909565 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.911146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.911764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.912223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.913196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.915893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.928307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b598a1-8e22-4422-86fa-a335161ebf1d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.929811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1090/2162] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:15.931442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.933624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:15.949920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b47ca-ec90-4c6b-8388-db35752d7c93 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.952027 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.953757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.954547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.954687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:15.955575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.955777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:15.956156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:15.956529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] PATH_INFO: `/v3/projects/d5e72842c30a4d8e976f03a16ebac257` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:15.957072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:15.957647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.957974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e72842c30a4d8e976f03a16ebac257)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:15.960076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.960816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:15.976403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1073/2163] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:29:15.977600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:15.994912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e055d-e889-4010-a42f-36b0aacf746d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:15.999697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.000601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.000881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.001166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] PATH_INFO: `/v3/projects/6a6212f207de4eaa9a89876e507d3d16/users/9879899651954bd49c0fbaf26f3acbd4/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.020797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6212f207de4eaa9a89876e507d3d16, user_id=9879899651954bd49c0fbaf26f3acbd4, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.023581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.108549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1074/2164] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:15 2022] PUT /identity/v3/projects/6a6212f207de4eaa9a89876e507d3d16/users/9879899651954bd49c0fbaf26f3acbd4/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:16.109527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.123140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e72842c30a4d8e976f03a16ebac257', 'request_id': 'req-773a83e3-392e-4ac5-b21d-b51f2554295c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:16.129142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e72842c30a4d8e976f03a16ebac257', 'request_id': 'req-773a83e3-392e-4ac5-b21d-b51f2554295c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:16.174663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a3db2-5c35-413a-8643-b5bda28a0ae7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.224135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e72842c30a4d8e976f03a16ebac257', 'request_id': 'req-773a83e3-392e-4ac5-b21d-b51f2554295c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:16.236790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da84ff-507c-494a-9b34-e1da98657737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.237477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da84ff-507c-494a-9b34-e1da98657737 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.237774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da84ff-507c-494a-9b34-e1da98657737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.238163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da84ff-507c-494a-9b34-e1da98657737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.239774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da84ff-507c-494a-9b34-e1da98657737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.240660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] The token cache is being invalidate because project d5e72842c30a4d8e976f03a16ebac257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:16.241130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:16.241841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da84ff-507c-494a-9b34-e1da98657737 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.242186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:16.244684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1091/2165] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:15 2022] DELETE /identity/v3/projects/d5e72842c30a4d8e976f03a16ebac257 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:16.246119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.251702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:16.269455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:16.274177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a83e3-392e-4ac5-b21d-b51f2554295c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.278924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd916a9-267b-4fd9-9fe0-9b7ebde5088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.279627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd916a9-267b-4fd9-9fe0-9b7ebde5088e admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.279912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd916a9-267b-4fd9-9fe0-9b7ebde5088e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.280192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd916a9-267b-4fd9-9fe0-9b7ebde5088e admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.280908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd916a9-267b-4fd9-9fe0-9b7ebde5088e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.282515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd916a9-267b-4fd9-9fe0-9b7ebde5088e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.293742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1092/2166] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:16.295151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.295461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.295743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.315028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:16.316283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:16.325037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1093/2167] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:16.325878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.336237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58025bb6-a6b1-4853-bec0-3313cff3333c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.341115 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc542b-c3e0-4204-b793-2ef336178bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.342019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc542b-c3e0-4204-b793-2ef336178bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.342299 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc542b-c3e0-4204-b793-2ef336178bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.342571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc542b-c3e0-4204-b793-2ef336178bb6 admin admin] PATH_INFO: `/v3/projects/6a6212f207de4eaa9a89876e507d3d16/users/9879899651954bd49c0fbaf26f3acbd4/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.344628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc542b-c3e0-4204-b793-2ef336178bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6212f207de4eaa9a89876e507d3d16, user_id=9879899651954bd49c0fbaf26f3acbd4, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.347388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc542b-c3e0-4204-b793-2ef336178bb6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.359170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1094/2168] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:16 2022] PUT /identity/v3/projects/6a6212f207de4eaa9a89876e507d3d16/users/9879899651954bd49c0fbaf26f3acbd4/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:16.361287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.361287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.361287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.381689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:16.382897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:16.431653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1075/2169] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/projects => generated 373 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:16.431653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.431653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.431653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.467632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] MFA Rules not processed for user `9879899651954bd49c0fbaf26f3acbd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:16.468989 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:16.514285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1095/2170] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:16.515381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.516682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:16.532361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab8258-53e3-4b59-9e58-66f16fb2bb8e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.559500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1076/2171] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:16.560611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.605315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.605935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.606219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.606497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.607186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.609688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.639960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9156b-f98a-48d7-a829-4b4d768625b9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.644848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.648759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.648759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.648759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.648759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:16.648759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.649756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.662833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1077/2172] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:29:16.663988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.665772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:16.738587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1096/2173] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:16.738587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.761408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6245d2-0844-4368-a310-8da4882b021e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.767001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.768068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.768454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.768805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.769742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.773033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.773605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893d07-273f-4c90-b225-c76e256775de admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.778014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.778752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.779389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.779846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.792458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.794214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.811333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1078/2174] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:16.812776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.828957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede61f-a278-4bb1-ac26-4956682a4640 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.842714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.843579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.843933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.850003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.850003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.850003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.862070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1079/2175] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:16.866632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.878621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94879d-c967-4f70-9f55-090b2048fd86 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.884568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.886117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.886556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.886988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.887766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.889447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:16.903850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1080/2176] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:16.907678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.923911 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659e731-fb82-4398-a73b-91248197a933 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:16.931053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fee29a-0ab8-462a-a089-389b1323c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:16.931987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee29a-0ab8-462a-a089-389b1323c5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:16.932271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee29a-0ab8-462a-a089-389b1323c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:16.932552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee29a-0ab8-462a-a089-389b1323c5d7 admin admin] PATH_INFO: `/v3/projects/bbeccdd72c294c54bfcd0a892a7054d7/users/702b231efa384c49aa03c7a5ab198da9/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:16.939607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1097/2177] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:16.941999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:16.949091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee29a-0ab8-462a-a089-389b1323c5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbeccdd72c294c54bfcd0a892a7054d7, user_id=702b231efa384c49aa03c7a5ab198da9, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:16.952382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee29a-0ab8-462a-a089-389b1323c5d7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.015629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b825a-23f2-42f3-bbbe-8de2e67898c4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.021566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.022317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.022604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.023059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.023809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.025764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.135550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1081/2178] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:16 2022] PUT /identity/v3/projects/bbeccdd72c294c54bfcd0a892a7054d7/users/702b231efa384c49aa03c7a5ab198da9/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:17.139482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.139930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.140340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.158679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1098/2179] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:16 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:29:17.160128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:17.164004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:17.208229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] MFA Rules not processed for user `702b231efa384c49aa03c7a5ab198da9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:17.215167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:17.290489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edc64a-882f-4eb6-874c-7d4c8a926fc0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.353628 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1082/2180] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:17.355754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:17.387669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.388657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.389123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.389588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.393423 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:17.440263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:17.456213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e7390-4619-46f7-9aae-9bd9c0a538e7 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.463235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.464103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.464482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.464855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.465684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.468132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.470064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.471675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.482199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1083/2181] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:17.482199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:17.486113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1099/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:17.487183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:17.503348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f03eb5-f9e0-494a-bc62-2d2de06fed29 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.510375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.511654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc3dd1-8175-42bf-936c-47df784e1c0e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.512304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.512736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.514011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.514156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.515967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.520897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199df4e7-af62-4819-ab13-c1a5012a3429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.523869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199df4e7-af62-4819-ab13-c1a5012a3429 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.524930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199df4e7-af62-4819-ab13-c1a5012a3429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.525620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199df4e7-af62-4819-ab13-c1a5012a3429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.532432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1100/2183] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:17.533563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:17.547060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199df4e7-af62-4819-ab13-c1a5012a3429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.549174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199df4e7-af62-4819-ab13-c1a5012a3429 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.549312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3d2c2-2745-465f-99ca-35a7ace7564a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.557993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.559097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.559460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.559745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] PATH_INFO: `/v3/projects/d4d14f6b545a400f89085aef87e11467/users/3f142158233d426481a78e09caf089b4/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.572550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1084/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:17.585376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d14f6b545a400f89085aef87e11467, user_id=3f142158233d426481a78e09caf089b4, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.590223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.685223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1101/2185] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:17 2022] PUT /identity/v3/projects/d4d14f6b545a400f89085aef87e11467/users/3f142158233d426481a78e09caf089b4/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:17.697078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62212-e2d2-4a2c-99f9-483b2a82135f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.697454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62212-e2d2-4a2c-99f9-483b2a82135f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.697740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e62212-e2d2-4a2c-99f9-483b2a82135f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.733572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d3e62212-e2d2-4a2c-99f9-483b2a82135f None None] MFA Rules not processed for user `3f142158233d426481a78e09caf089b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:17.735235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d3e62212-e2d2-4a2c-99f9-483b2a82135f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:17.851681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1085/2186] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:17.864317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:17.872439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:17.948619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0384d0-cde0-4914-8363-79134c497683 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:17.968761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:17.970509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:17.971213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:17.971536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:17.971814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:17.984582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:17.986224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:17.996950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1102/2187] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:18.484210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee82ea8-4ac1-41ae-88d9-36aa3d0e0491 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:18.484210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee82ea8-4ac1-41ae-88d9-36aa3d0e0491 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:18.484210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee82ea8-4ac1-41ae-88d9-36aa3d0e0491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:18.517635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-eee82ea8-4ac1-41ae-88d9-36aa3d0e0491 None None] MFA Rules not processed for user `a0a147daff9846e699c26259fac4e004`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:18.519333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-eee82ea8-4ac1-41ae-88d9-36aa3d0e0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:18.583891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1086/2188] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:18.605354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:19.174970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:19.468083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:20.129395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:20.368904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:20.587979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:23.796251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:23.802773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:23.893592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97512dc-d14b-4798-9df7-1623837b8af4 service glance] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:23.898848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:23.899957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:23.900403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:23.900794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] PATH_INFO: `/v3/users/87b7124a042e476992a1b9ba82cf19f3` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:23.902208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b7124a042e476992a1b9ba82cf19f3)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:23.904483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:24.210808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b7124a042e476992a1b9ba82cf19f3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.211996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b7124a042e476992a1b9ba82cf19f3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.227173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b7124a042e476992a1b9ba82cf19f3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.236048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b7124a042e476992a1b9ba82cf19f3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.350042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b7124a042e476992a1b9ba82cf19f3'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.360222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1103/2189] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:23 2022] DELETE /identity/v3/users/87b7124a042e476992a1b9ba82cf19f3 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:24.365158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:24.365632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:24.365711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:24.368891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:24.370371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:24.454487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:24.456477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:24.458471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:24.485533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c2516-7320-4fc4-869b-57902fd56216 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:24.488523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:24.489953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:24.490495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:24.490495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] PATH_INFO: `/v3/users/702b231efa384c49aa03c7a5ab198da9` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:24.491969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] RBAC: Authorizing `identity:delete_user(user_id=702b231efa384c49aa03c7a5ab198da9)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:24.494012 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:24.556307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1087/2190] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:24.567642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:24.569713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:24.599239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:24.687949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8571da-e8f2-4ec5-9ccd-e45e33747f0d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:24.692681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a1b75-b5ee-414e-aca8-435d44c80b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:24.694254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1b75-b5ee-414e-aca8-435d44c80b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:24.694805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1b75-b5ee-414e-aca8-435d44c80b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:24.694805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a1b75-b5ee-414e-aca8-435d44c80b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:24.716617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a1b75-b5ee-414e-aca8-435d44c80b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:24.719377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a1b75-b5ee-414e-aca8-435d44c80b14 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:24.724179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702b231efa384c49aa03c7a5ab198da9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.725110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702b231efa384c49aa03c7a5ab198da9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.731253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1088/2191] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:29:24.740259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702b231efa384c49aa03c7a5ab198da9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.749402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702b231efa384c49aa03c7a5ab198da9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.837598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702b231efa384c49aa03c7a5ab198da9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:24.844952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1104/2192] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:24 2022] DELETE /identity/v3/users/702b231efa384c49aa03c7a5ab198da9 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:24.850048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:24.850491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:24.850857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:24.871895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:24.871895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:24.975736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1089/2193] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:25.002270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:25.008419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:25.127430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765da9b3-6674-425c-b21e-29f1a9ff034a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:25.136166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:25.137073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:25.137456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:25.137828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:25.184613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:25.189669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:25.212859 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1105/2194] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:25.463134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:25.465017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:25.492107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ac86b-9a91-472a-b8f3-2d3406535e30 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:25.502537 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:25.503557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:25.503848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:25.504139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] PATH_INFO: `/v3/projects/09cba523ef8e459db613b4f4ed5d4093` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:25.507180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cba523ef8e459db613b4f4ed5d4093)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:25.509371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:25.632064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cba523ef8e459db613b4f4ed5d4093', 'request_id': 'req-aa1da5fa-1331-45cf-a63e-c8870d8d835e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:25.700786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cba523ef8e459db613b4f4ed5d4093', 'request_id': 'req-aa1da5fa-1331-45cf-a63e-c8870d8d835e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:25.708654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cba523ef8e459db613b4f4ed5d4093', 'request_id': 'req-aa1da5fa-1331-45cf-a63e-c8870d8d835e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:25.721828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] The token cache is being invalidate because project 09cba523ef8e459db613b4f4ed5d4093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:25.722141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:25.722781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:25.724904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1090/2195] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:25 2022] DELETE /identity/v3/projects/09cba523ef8e459db613b4f4ed5d4093 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:25.729518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:25.741167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f616-d251-48b5-9fb5-40e42451035c service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:25.801944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:25.803102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:25.803534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:25.803918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] PATH_INFO: `/v3/users/9879899651954bd49c0fbaf26f3acbd4` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:25.805308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9879899651954bd49c0fbaf26f3acbd4)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:25.807404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:25.810274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:26.024405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:26.026334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:26.037383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9879899651954bd49c0fbaf26f3acbd4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.039045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9879899651954bd49c0fbaf26f3acbd4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.053249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1da5fa-1331-45cf-a63e-c8870d8d835e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:26.058815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9879899651954bd49c0fbaf26f3acbd4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.064804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:26.068267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.068620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9879899651954bd49c0fbaf26f3acbd4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.069029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.069550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] PATH_INFO: `/v3/projects/bbeccdd72c294c54bfcd0a892a7054d7` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.072940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbeccdd72c294c54bfcd0a892a7054d7)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:26.076112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:26.095658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:26.171528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-daaae9e7-e0fa-4718-9021-41065e4016a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9879899651954bd49c0fbaf26f3acbd4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.182234 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1106/2196] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:25 2022] DELETE /identity/v3/users/9879899651954bd49c0fbaf26f3acbd4 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:26.281129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbeccdd72c294c54bfcd0a892a7054d7', 'request_id': 'req-a5fbe94d-f849-459f-a9ef-f600b3f24a81'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.400358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbeccdd72c294c54bfcd0a892a7054d7', 'request_id': 'req-a5fbe94d-f849-459f-a9ef-f600b3f24a81'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.407045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbeccdd72c294c54bfcd0a892a7054d7', 'request_id': 'req-a5fbe94d-f849-459f-a9ef-f600b3f24a81'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.425605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] The token cache is being invalidate because project bbeccdd72c294c54bfcd0a892a7054d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:26.425949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.427940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a5fbe94d-f849-459f-a9ef-f600b3f24a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:26.435558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1091/2197] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:26 2022] DELETE /identity/v3/projects/bbeccdd72c294c54bfcd0a892a7054d7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:26.437091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.437397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.437871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.484091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:26.485530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:26.555429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1107/2198] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:26.563709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.564186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.564186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.589299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:26.591506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:26.602469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1092/2199] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:26.607526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:26.623580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aaeea-8cff-4b1c-a088-17a4fe373631 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:26.632873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:26.634166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.634556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.635123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.635705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:26.636021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:26.638019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:26.655357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1108/2200] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:29:26.660216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:26.670953 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18ac35-5379-4ba3-bf97-71dca06c06ab None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:26.677014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:26.677939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.678241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.678505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.679204 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:26.681236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:26.787345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1093/2201] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:26 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:26.789191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:26.800467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc01f5-5a59-41b0-b6c3-487ac8183de9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:26.864538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:26.865211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.865495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.865772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.866752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:26.868749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:26.887664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:26.891695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:26.918472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47612a09-abc4-43f7-a30d-8a34f50e8d16 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:26.924429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:26.925399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.925755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.926361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] PATH_INFO: `/v3/projects/6a6212f207de4eaa9a89876e507d3d16` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.927795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6212f207de4eaa9a89876e507d3d16)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:26.929946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:26.969951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1109/2202] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:26 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:29:26.971123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:26.981969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd7df4-3e9d-4979-b8f8-dba403c2c54c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:26.987125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:26.987890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:26.988178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:26.988576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:26.989216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:26.991497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:26.995856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:27.007331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1110/2203] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:29:27.008634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.018942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda501e-a6d5-4115-aa97-22ca58a400ad admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.024490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.025292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.025538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.025823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.026552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.028252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:27.039961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1111/2204] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:27.044024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.054488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a8616-332b-44f9-982b-f0805b2d08e0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.062929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b594dd7-b6e3-48a0-b7dc-b1f5e34ab104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.063900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594dd7-b6e3-48a0-b7dc-b1f5e34ab104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.064190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594dd7-b6e3-48a0-b7dc-b1f5e34ab104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.064500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594dd7-b6e3-48a0-b7dc-b1f5e34ab104 admin admin] PATH_INFO: `/v3/projects/e2080f03ddd148889160f6f51aa71955/users/be19da315701413e810d6e1274b36f2b/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.082137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594dd7-b6e3-48a0-b7dc-b1f5e34ab104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2080f03ddd148889160f6f51aa71955, user_id=be19da315701413e810d6e1274b36f2b, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.085088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594dd7-b6e3-48a0-b7dc-b1f5e34ab104 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:27.088063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6212f207de4eaa9a89876e507d3d16', 'request_id': 'req-e27de180-e6f7-4e5a-906f-656e88902718'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:27.241207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1112/2205] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:27 2022] PUT /identity/v3/projects/e2080f03ddd148889160f6f51aa71955/users/be19da315701413e810d6e1274b36f2b/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:27.252287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.252576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.252875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.268455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6212f207de4eaa9a89876e507d3d16', 'request_id': 'req-e27de180-e6f7-4e5a-906f-656e88902718'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:27.288154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] MFA Rules not processed for user `be19da315701413e810d6e1274b36f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:27.288869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6212f207de4eaa9a89876e507d3d16', 'request_id': 'req-e27de180-e6f7-4e5a-906f-656e88902718'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:27.289805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:27.315436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] The token cache is being invalidate because project 6a6212f207de4eaa9a89876e507d3d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:27.315736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:27.316451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e27de180-e6f7-4e5a-906f-656e88902718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:27.322401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1094/2206] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:26 2022] DELETE /identity/v3/projects/6a6212f207de4eaa9a89876e507d3d16 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:27.324001 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f1b7-9ba0-46ba-b7a8-beb6fc6593b6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.324409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f1b7-9ba0-46ba-b7a8-beb6fc6593b6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.324691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f1b7-9ba0-46ba-b7a8-beb6fc6593b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.358358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5774f1b7-9ba0-46ba-b7a8-beb6fc6593b6 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:27.359880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5774f1b7-9ba0-46ba-b7a8-beb6fc6593b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:27.408497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1113/2207] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:27.423456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.427290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:27.471093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:27.532203 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1095/2208] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:27.542782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.547781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.548251 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.570226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7ce81-16fe-4e80-98c6-ac6509266294 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.576025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.577112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.577500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.577895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.590769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.592573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:27.606575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:27.610763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:27.620259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1114/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:29:27.644464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1096/2210] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:27.645970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.656765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.658372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:27.667029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8085c-61e5-44f6-b661-b9379e0264d8 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.682375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.684005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.684392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.684776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.685777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:27.686185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.687593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51450251-2658-4fc2-8fec-eabd5c26a401 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.688609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:27.709769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1115/2211] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:29:27.716298 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.726198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.727190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.740833 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.741113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740302-1130-4d6f-b865-ca3ccd32e472 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.741235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] PATH_INFO: `/v3/users/f030c1a8e0a84648848e1327c3f7e230` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.743639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f030c1a8e0a84648848e1327c3f7e230)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.748359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.749413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.750014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.750125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:27.750972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.752171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.754669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:27.869831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1116/2212] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:27 2022] POST /identity/v3/projects => generated 409 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:27.873618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:27.878204 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:27.951344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be3943-6d92-4232-a4f4-64b2a1f0e5dc admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:27.957127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:27.958478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:27.958914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:27.959187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:27.960009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:27.961958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.065090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f030c1a8e0a84648848e1327c3f7e230'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:28.073925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f030c1a8e0a84648848e1327c3f7e230'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:28.076680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f030c1a8e0a84648848e1327c3f7e230'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:28.085545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1117/2213] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:27 2022] POST /identity/v3/users => generated 305 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:29:28.086920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.090248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:28.095129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f030c1a8e0a84648848e1327c3f7e230'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:28.103438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c86e0-fba7-4b53-aae6-1fd5b07b583c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.108455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.109127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.109413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.109691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.110416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.112098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.123030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1118/2214] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:28.130230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.141353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fdab2-0d9c-4541-84eb-6e3ef2050e2b admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.147337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.148013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.148303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.148588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.149317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.150941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.164106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1119/2215] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:29:28.168741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.179737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014b61e-88b7-4c25-abdb-13963e5eb895 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.184462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f28f8e-354a-4997-8d08-b9905b53ccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.185333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f28f8e-354a-4997-8d08-b9905b53ccb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.185617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f28f8e-354a-4997-8d08-b9905b53ccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.185893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f28f8e-354a-4997-8d08-b9905b53ccb5 admin admin] PATH_INFO: `/v3/projects/3332b7f1a23249f8a157a783934ed667/users/02b429283c144508aee28efb65dca164/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.188019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e6582411-95e3-4289-a27f-02688f090fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f030c1a8e0a84648848e1327c3f7e230'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:28.195644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1097/2216] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:27 2022] DELETE /identity/v3/users/f030c1a8e0a84648848e1327c3f7e230 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:28.200644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.200971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.201248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.209804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f28f8e-354a-4997-8d08-b9905b53ccb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3332b7f1a23249f8a157a783934ed667, user_id=02b429283c144508aee28efb65dca164, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.212522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f28f8e-354a-4997-8d08-b9905b53ccb5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.220510 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:28.221949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:28.292543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1098/2217] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:28.302274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.303604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:28.309045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1120/2218] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:28 2022] PUT /identity/v3/projects/3332b7f1a23249f8a157a783934ed667/users/02b429283c144508aee28efb65dca164/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:28.321437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.321817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.322161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.360941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] MFA Rules not processed for user `02b429283c144508aee28efb65dca164`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:28.363460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:28.382050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c2fdc-72e4-4253-9b9a-6949399b00c4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.390531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:28.453265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.454234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.454637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.455042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.461946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1121/2219] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:28.466070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.516714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:28.545459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.545772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a12c9-ec4b-46af-a790-61bdec2cd16e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.546710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.550811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.553010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.553010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.553249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.553957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.556121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.562768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1099/2220] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:28.720895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1122/2221] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:28 2022] POST /identity/v3/projects => generated 409 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:29:28.726596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.758006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804595c-6044-41e3-9809-af106f912b52 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.821035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.821686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.822117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.822406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.823168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.825130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.898046 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1100/2222] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:28 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:28.901918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.924055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42676e8-d364-4403-b715-1142fa59d1f7 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.929434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.930114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.930391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.932856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.932856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.933238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:28.945055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1123/2223] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:28.949145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:28.960431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e46222-6b55-4d4d-b677-156606cbd690 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:28.965369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:28.966273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:28.966723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:28.967012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] PATH_INFO: `/v3/projects/70794a1f38db432a857e6771f794e003/users/39f36c120d184115b1f6658b880cde84/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:28.983627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70794a1f38db432a857e6771f794e003, user_id=39f36c120d184115b1f6658b880cde84, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:28.986538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.129892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1101/2224] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:28 2022] PUT /identity/v3/projects/70794a1f38db432a857e6771f794e003/users/39f36c120d184115b1f6658b880cde84/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:29.134604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:29.146361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a05f71-fe5e-4c93-8bbe-d80bfa5d9a26 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:29.185316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:29.186074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:29.233123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:29.237449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:29.238386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.239501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.239979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.240961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:29.242889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.255996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1124/2225] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:29.260284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:29.274097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d74b-bd6f-4f78-9cc4-bcb6fc9a7d05 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:29.279193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641aa7a-e6f0-4f9d-9f55-6b45d34a5119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:29.280006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641aa7a-e6f0-4f9d-9f55-6b45d34a5119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.280295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641aa7a-e6f0-4f9d-9f55-6b45d34a5119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.280773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641aa7a-e6f0-4f9d-9f55-6b45d34a5119 admin admin] PATH_INFO: `/v3/projects/70794a1f38db432a857e6771f794e003/users/39f36c120d184115b1f6658b880cde84/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.282690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641aa7a-e6f0-4f9d-9f55-6b45d34a5119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70794a1f38db432a857e6771f794e003, user_id=39f36c120d184115b1f6658b880cde84, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:29.285648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641aa7a-e6f0-4f9d-9f55-6b45d34a5119 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.297243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1125/2226] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:29 2022] PUT /identity/v3/projects/70794a1f38db432a857e6771f794e003/users/39f36c120d184115b1f6658b880cde84/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:29.308931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.309376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.310332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.322373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983baee-d8bd-4cbc-88d0-72f40eeb0959 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:29.348819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] MFA Rules not processed for user `39f36c120d184115b1f6658b880cde84`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:29.350050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:29.387402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:29.388416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.388775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.389129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] PATH_INFO: `/v3/projects/607e5f7daaa24217ba9f830e05aa2015` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.390324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=607e5f7daaa24217ba9f830e05aa2015)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:29.392339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.429489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1126/2227] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:29.440213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:29.441512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:29.452190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:29.502218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '607e5f7daaa24217ba9f830e05aa2015', 'request_id': 'req-642ae613-fb1e-4e42-8ae2-6ec73c476584'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:29.510599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a31ea-d15d-4cb2-a44b-94927a28bc7b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:29.561871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def5fc-d243-4b1f-acc0-19968b446bc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:29.562524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def5fc-d243-4b1f-acc0-19968b446bc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.562814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def5fc-d243-4b1f-acc0-19968b446bc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.563093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def5fc-d243-4b1f-acc0-19968b446bc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.603837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '607e5f7daaa24217ba9f830e05aa2015', 'request_id': 'req-642ae613-fb1e-4e42-8ae2-6ec73c476584'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:29.605962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '607e5f7daaa24217ba9f830e05aa2015', 'request_id': 'req-642ae613-fb1e-4e42-8ae2-6ec73c476584'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:29.619674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] The token cache is being invalidate because project 607e5f7daaa24217ba9f830e05aa2015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:29.620071 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:29.620703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-642ae613-fb1e-4e42-8ae2-6ec73c476584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:29.626192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1102/2228] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:29 2022] DELETE /identity/v3/projects/607e5f7daaa24217ba9f830e05aa2015 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:29.628188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.628554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.628894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.648721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54def5fc-d243-4b1f-acc0-19968b446bc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:29.650327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54def5fc-d243-4b1f-acc0-19968b446bc5 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.662041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1127/2229] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:29:29.669515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:29.670870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:29.714894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:29.746126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1103/2230] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:29:29.759044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.759044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.759044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.777392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:29.778625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:29.786878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1128/2231] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:29.791822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:29.802146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4ba80-5ffc-4905-972d-115c9df65654 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:29.806934 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:29.807705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.807983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.808269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.809070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:29.809393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:29.811121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.827636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1104/2232] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:29:29.828754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:29.840475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a2a-ab5f-4808-81db-9a4a7a5a9bc1 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:29.845567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:29.847262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:29.847262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:29.847262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:29.847635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:29.849720 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:29.996155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1129/2233] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:29:29.999898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:30.010731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a3c4d-b3c9-40db-a7ad-a4f89881095e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:30.086488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:30.087186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.087508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.087791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.088511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:30.090500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:30.154551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:30.156514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:30.206177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1105/2234] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:29 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:30.210741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:30.221899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f775e-05bf-492b-ace3-a5bc206c328f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:30.225375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:30.227453 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:30.228128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.228410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.228691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.229416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:30.231054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:30.231456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed662d-e57a-4179-b0b7-9ced6bbd9965 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:30.235688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:30.237625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.237625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.237625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.239632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:30.242142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1106/2235] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:30.247499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:30.260572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643861c-bf3e-4289-96d1-e59ae8487ed4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:30.274244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:30.274942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.275225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.275542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.276287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:30.277906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:30.293339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1107/2236] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:29:30.297680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:30.306075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:30.307784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2570000-e02d-4376-8090-b6ef5c6eefec service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:30.310999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8fac9-e1a0-41b7-96ed-60307856292e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:30.317529 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69b913-49a2-40df-aeb3-afaad79ec010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:30.318590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69b913-49a2-40df-aeb3-afaad79ec010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.318891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69b913-49a2-40df-aeb3-afaad79ec010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.318993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1130/2237] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:30.319401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69b913-49a2-40df-aeb3-afaad79ec010 admin admin] PATH_INFO: `/v3/projects/57b4fd759f424c4c91d0ceb2e3b60485/users/e03304ac49694a0aa3b779ca368b6dbf/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.337098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69b913-49a2-40df-aeb3-afaad79ec010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b4fd759f424c4c91d0ceb2e3b60485, user_id=e03304ac49694a0aa3b779ca368b6dbf, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:30.339935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69b913-49a2-40df-aeb3-afaad79ec010 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:30.446421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1108/2238] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:30 2022] PUT /identity/v3/projects/57b4fd759f424c4c91d0ceb2e3b60485/users/e03304ac49694a0aa3b779ca368b6dbf/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:30.456413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.456712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.456985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.491623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] MFA Rules not processed for user `e03304ac49694a0aa3b779ca368b6dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:30.492860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:30.587508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.587992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1131/2239] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:30.588135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.588514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.605231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:30.606187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:30.614420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] MFA Rules not processed for user `be19da315701413e810d6e1274b36f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:30.617445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:30.678682 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053f24f-e872-44d6-9c6d-3a1453513e04 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:30.682823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fee0d7-d6c7-4397-96e1-f56a3db10d7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:30.683508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fee0d7-d6c7-4397-96e1-f56a3db10d7b service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:30.683793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fee0d7-d6c7-4397-96e1-f56a3db10d7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:30.684080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fee0d7-d6c7-4397-96e1-f56a3db10d7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:30.685446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1109/2240] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:30.696186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fee0d7-d6c7-4397-96e1-f56a3db10d7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:30.698394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fee0d7-d6c7-4397-96e1-f56a3db10d7b service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:30.710119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1132/2241] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:29:40.484640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:40.498098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:40.564156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d9479-269f-48b2-88d5-44af0aae3941 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:40.572293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:40.573200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:40.573484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:40.574022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] PATH_INFO: `/v3/users/e03304ac49694a0aa3b779ca368b6dbf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:40.575033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e03304ac49694a0aa3b779ca368b6dbf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:40.577908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:40.784921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e03304ac49694a0aa3b779ca368b6dbf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:40.792879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e03304ac49694a0aa3b779ca368b6dbf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:40.794309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e03304ac49694a0aa3b779ca368b6dbf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:40.816125 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e03304ac49694a0aa3b779ca368b6dbf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:40.891470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e03304ac49694a0aa3b779ca368b6dbf'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:40.902658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1110/2242] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:40 2022] DELETE /identity/v3/users/e03304ac49694a0aa3b779ca368b6dbf => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:40.914516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:40.915454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:40.915912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:40.941088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:40.943183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:40.992713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:41.048453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1133/2243] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:41.060527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:41.062348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:41.142188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee45a7f-070a-475d-84c4-9f9ac03153f7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:41.149124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d3a9-f19b-43af-92c5-c3af6212d675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:41.150238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d3a9-f19b-43af-92c5-c3af6212d675 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:41.150556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d3a9-f19b-43af-92c5-c3af6212d675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:41.150852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d3a9-f19b-43af-92c5-c3af6212d675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:41.162702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d3a9-f19b-43af-92c5-c3af6212d675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:41.164255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d3a9-f19b-43af-92c5-c3af6212d675 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:41.181138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1111/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:41.969101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:41.970600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:41.993595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde227f8-1518-4f41-b205-f81a3bde60fa None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.024878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.025789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.026246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.026690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] PATH_INFO: `/v3/projects/57b4fd759f424c4c91d0ceb2e3b60485` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.028613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b4fd759f424c4c91d0ceb2e3b60485)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.031996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:42.146817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b4fd759f424c4c91d0ceb2e3b60485', 'request_id': 'req-13fbad61-f9d9-4543-82c5-193274e12ce9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:42.155776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b4fd759f424c4c91d0ceb2e3b60485', 'request_id': 'req-13fbad61-f9d9-4543-82c5-193274e12ce9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:42.198724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b4fd759f424c4c91d0ceb2e3b60485', 'request_id': 'req-13fbad61-f9d9-4543-82c5-193274e12ce9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:42.211902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] The token cache is being invalidate because project 57b4fd759f424c4c91d0ceb2e3b60485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:42.212191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:42.212870 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-13fbad61-f9d9-4543-82c5-193274e12ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:42.214409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1134/2245] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:41 2022] DELETE /identity/v3/projects/57b4fd759f424c4c91d0ceb2e3b60485 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:42.220750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.221216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.222029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.271505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:42.277958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:42.333945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:42.373058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1112/2246] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:42.377283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.377573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.377858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.401699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:42.403875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:42.420137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1135/2247] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:42.421976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:42.446904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f670b50-e332-4a0b-8388-8458640267d5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.453285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.454081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.455026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.455469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.456336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:42.456749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.459267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:42.475067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1113/2248] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:29:42.476357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:42.491448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43489c-d566-40ac-977a-2619e6a6680e None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.497188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.497845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.498136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.498601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.509816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.513896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:42.617807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1136/2249] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:42 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:42.623192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:42.638585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1e9ef-542e-4cc2-becc-fe61a3d4b1e1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.717175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.718024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.718313 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.718596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.719293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.721554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:42.812851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1114/2250] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:42 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:42.818971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:42.831357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35bafb-559e-4d9f-aab3-70720a5f1ff8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.836712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.837386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.837673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.837951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.838684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.841458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:42.864901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1137/2251] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:42.869797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:42.881562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eebfa-a0f8-47dd-9932-5f6c404f3333 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.888085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b3d8c-e539-4da1-8fa5-e2289f92af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.889235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3d8c-e539-4da1-8fa5-e2289f92af4d admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.889533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3d8c-e539-4da1-8fa5-e2289f92af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.889816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3d8c-e539-4da1-8fa5-e2289f92af4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.890559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3d8c-e539-4da1-8fa5-e2289f92af4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.892340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3d8c-e539-4da1-8fa5-e2289f92af4d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:42.911508 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1115/2252] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:42.921251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:42.934671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c764427-4d84-4ede-88f6-6057db1db052 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:42.940106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:42.941389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:42.941802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:42.942169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] PATH_INFO: `/v3/projects/9f84c80ebf3c492baf85c4720ade1090/users/e4a9e14a3aff4da1a2e31d4f8948dd59/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:42.963347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f84c80ebf3c492baf85c4720ade1090, user_id=e4a9e14a3aff4da1a2e31d4f8948dd59, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:42.970172 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:43.041497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1138/2253] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:42 2022] PUT /identity/v3/projects/9f84c80ebf3c492baf85c4720ade1090/users/e4a9e14a3aff4da1a2e31d4f8948dd59/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:43.054943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:43.055241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:43.055555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:43.086069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] MFA Rules not processed for user `e4a9e14a3aff4da1a2e31d4f8948dd59`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:43.088167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:43.170697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1116/2254] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:43.182953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:43.185283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:43.257564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024940-ebf0-4b3f-8d06-379c18e536f2 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:43.261257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd0c96-6739-405b-8f27-d9b551bfccdc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:43.261899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0c96-6739-405b-8f27-d9b551bfccdc service glance] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:43.262180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0c96-6739-405b-8f27-d9b551bfccdc service glance] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:43.262451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0c96-6739-405b-8f27-d9b551bfccdc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:43.268937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:43.272478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd0c96-6739-405b-8f27-d9b551bfccdc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:43.274265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd0c96-6739-405b-8f27-d9b551bfccdc service glance] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:43.284899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1139/2255] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:44.387958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:44.389338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:44.429158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:44.467293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b35bc-b14f-4037-9ec9-d35715b15be6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:44.473472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:44.474379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:44.474662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:44.474941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] PATH_INFO: `/v3/users/02b429283c144508aee28efb65dca164` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:44.476199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b429283c144508aee28efb65dca164)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:44.478138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:44.690193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b429283c144508aee28efb65dca164'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:44.699015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b429283c144508aee28efb65dca164'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:44.699831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b429283c144508aee28efb65dca164'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:44.719559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b429283c144508aee28efb65dca164'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:44.793667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b429283c144508aee28efb65dca164'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:44.804222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1117/2256] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:44 2022] DELETE /identity/v3/users/02b429283c144508aee28efb65dca164 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:44.809044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:44.809699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:44.809699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:44.830506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:44.832645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:44.929220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1140/2257] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:44.939534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:44.941747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:45.091562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e0ed0-e648-4eba-b905-2f38bd932bf0 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:45.105688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:45.106377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:45.106662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:45.106940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:45.123925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:45.125805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:45.142512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1118/2258] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:45.864728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:45.866709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:45.889553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a580afc-fbd1-4597-b46f-a4d8b863abeb None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:45.897990 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:45.899257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:45.899724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:45.900078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] PATH_INFO: `/v3/projects/3332b7f1a23249f8a157a783934ed667` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:45.901968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3332b7f1a23249f8a157a783934ed667)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:45.904281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:46.022278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3332b7f1a23249f8a157a783934ed667', 'request_id': 'req-cfd96b69-0422-4316-a79c-202f9f002d03'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.033318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3332b7f1a23249f8a157a783934ed667', 'request_id': 'req-cfd96b69-0422-4316-a79c-202f9f002d03'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.106088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3332b7f1a23249f8a157a783934ed667', 'request_id': 'req-cfd96b69-0422-4316-a79c-202f9f002d03'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.120381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] The token cache is being invalidate because project 3332b7f1a23249f8a157a783934ed667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:46.120706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.121341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.122895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1141/2259] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:45 2022] DELETE /identity/v3/projects/3332b7f1a23249f8a157a783934ed667 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:46.128502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:46.144541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7b0eb-6f67-42ed-aab1-159dad755ac9 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:46.201679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:46.211321 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:46.212246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:46.212533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:46.212811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] PATH_INFO: `/v3/users/39f36c120d184115b1f6658b880cde84` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:46.214094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f36c120d184115b1f6658b880cde84)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:46.216078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:46.432885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f36c120d184115b1f6658b880cde84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.439927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f36c120d184115b1f6658b880cde84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.440702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f36c120d184115b1f6658b880cde84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.454758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f36c120d184115b1f6658b880cde84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.518842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9084795f-509b-47b8-a0ad-c6501ac15fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f36c120d184115b1f6658b880cde84'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:46.526457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1119/2260] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:46 2022] DELETE /identity/v3/users/39f36c120d184115b1f6658b880cde84 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:47.124101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:47.134274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:47.140778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96b69-0422-4316-a79c-202f9f002d03 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:47.212211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:47.213246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.213526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.213801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] PATH_INFO: `/v3/projects/70794a1f38db432a857e6771f794e003` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.215202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70794a1f38db432a857e6771f794e003)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:47.217209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:47.243389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:47.314518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70794a1f38db432a857e6771f794e003', 'request_id': 'req-180a7d10-dad7-495d-b0d0-3d526f04cf2c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:47.321421 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70794a1f38db432a857e6771f794e003', 'request_id': 'req-180a7d10-dad7-495d-b0d0-3d526f04cf2c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:47.410597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70794a1f38db432a857e6771f794e003', 'request_id': 'req-180a7d10-dad7-495d-b0d0-3d526f04cf2c'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:47.423654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] The token cache is being invalidate because project 70794a1f38db432a857e6771f794e003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:47.423946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:47.424608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-180a7d10-dad7-495d-b0d0-3d526f04cf2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:47.426800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1142/2261] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:47 2022] DELETE /identity/v3/projects/70794a1f38db432a857e6771f794e003 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:47.432986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.433404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.433704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.466826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:47.468054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:47.530031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1120/2262] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:47.539168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.539496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.539786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.558907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:47.560342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:47.568600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1143/2263] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:47.572867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:47.582470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417365d-aacf-4f5c-bb45-e226f6160062 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:47.587040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:47.587723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.588012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.588298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.589025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:47.589335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:47.590935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:47.604649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1121/2264] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:29:47.609282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:47.619232 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e579616-0cd0-4bda-a1fe-788f8c0417c2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:47.624432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:47.625228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.625745 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.625957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.626690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:47.628860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:47.730992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1144/2265] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:47 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:47.735765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:47.747946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719df316-6f9d-4aa2-90bb-79cb83d4f3f0 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:47.797636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:47.809792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:47.810472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.810758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.811897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.811897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:47.814241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:47.937350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1122/2266] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:47 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:47.944114 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:47.966346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312d774-1177-40f8-97bf-b393a4aa2ff0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:47.976316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:47.977015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:47.986013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:47.986394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:47.987182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:47.989044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:48.003462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1145/2267] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:48.007243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:48.022400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683008d0-8775-4a82-8f58-1a5771f00ace admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:48.032574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9dcc9-37a1-4970-98d5-fa7e0a0ad574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:48.033709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9dcc9-37a1-4970-98d5-fa7e0a0ad574 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:48.034226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9dcc9-37a1-4970-98d5-fa7e0a0ad574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:48.034738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9dcc9-37a1-4970-98d5-fa7e0a0ad574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:48.037079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9dcc9-37a1-4970-98d5-fa7e0a0ad574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:48.040328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9dcc9-37a1-4970-98d5-fa7e0a0ad574 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:48.058454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1123/2268] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:48.059709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:48.071363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f265d-14b8-4f5e-8150-4be401d7c882 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:48.081823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:48.081823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:48.081823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:48.081823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] PATH_INFO: `/v3/projects/020f959e6c9942dda641d2488c175270/users/2ea165e3847b4c728d81aec0c3a13619/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:48.099392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020f959e6c9942dda641d2488c175270, user_id=2ea165e3847b4c728d81aec0c3a13619, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:48.102346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:48.201275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1146/2269] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:48 2022] PUT /identity/v3/projects/020f959e6c9942dda641d2488c175270/users/2ea165e3847b4c728d81aec0c3a13619/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:48.214116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:48.214406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:48.214692 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:48.244656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] MFA Rules not processed for user `2ea165e3847b4c728d81aec0c3a13619`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:48.245873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:48.314831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1124/2270] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:48.326691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:48.328501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:48.392102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63e1cb-c7f6-41a9-9fb5-053b7ae67de9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:48.396452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad252bc8-ea1f-47c7-b92d-aedcf7172b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:48.397302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad252bc8-ea1f-47c7-b92d-aedcf7172b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:48.397584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad252bc8-ea1f-47c7-b92d-aedcf7172b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:48.397857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad252bc8-ea1f-47c7-b92d-aedcf7172b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:48.414532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad252bc8-ea1f-47c7-b92d-aedcf7172b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:48.416621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad252bc8-ea1f-47c7-b92d-aedcf7172b8a service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:48.428374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1147/2271] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:52.323363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:52.328113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:52.414288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7600e8-27b7-41f2-b9b8-dba00d9d55f5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:52.419732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:52.420842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:52.421134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:52.421419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] PATH_INFO: `/v3/users/3f142158233d426481a78e09caf089b4` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:52.423196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f142158233d426481a78e09caf089b4)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:52.425503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:52.641979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f142158233d426481a78e09caf089b4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:52.649700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f142158233d426481a78e09caf089b4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:52.650378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f142158233d426481a78e09caf089b4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:52.654908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:52.663443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f142158233d426481a78e09caf089b4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:52.742013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f142158233d426481a78e09caf089b4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:52.748649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1125/2272] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:29:52 2022] DELETE /identity/v3/users/3f142158233d426481a78e09caf089b4 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:52.753268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:52.754032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:52.754032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:52.777864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:52.779053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:52.835611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:52.851376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1148/2273] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:52.861197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:52.863392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:52.926500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19c455-b7da-40ba-9553-5f5011e69fb3 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:52.930877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3be0a9-b45c-4e46-b7d9-50c10de9d749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:52.931709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3be0a9-b45c-4e46-b7d9-50c10de9d749 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:52.932035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3be0a9-b45c-4e46-b7d9-50c10de9d749 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:52.932471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3be0a9-b45c-4e46-b7d9-50c10de9d749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:52.944175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3be0a9-b45c-4e46-b7d9-50c10de9d749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:52.945908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3be0a9-b45c-4e46-b7d9-50c10de9d749 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:52.959639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1126/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:29:53.536827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:53.538524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:53.553648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38ea8a-45ca-4cb8-8bb3-2ec49d57e93a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:53.560717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:53.561823 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:53.562165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:53.562438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] PATH_INFO: `/v3/projects/d4d14f6b545a400f89085aef87e11467` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:53.563839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d14f6b545a400f89085aef87e11467)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:53.566061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:53.714528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d14f6b545a400f89085aef87e11467', 'request_id': 'req-85b8c976-8111-4aa7-93cb-165e2c303f69'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:53.720416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d14f6b545a400f89085aef87e11467', 'request_id': 'req-85b8c976-8111-4aa7-93cb-165e2c303f69'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:53.816125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d14f6b545a400f89085aef87e11467', 'request_id': 'req-85b8c976-8111-4aa7-93cb-165e2c303f69'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:53.828761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] The token cache is being invalidate because project d4d14f6b545a400f89085aef87e11467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:29:53.829047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:53.829708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-85b8c976-8111-4aa7-93cb-165e2c303f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:29:53.831231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1149/2275] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:29:53 2022] DELETE /identity/v3/projects/d4d14f6b545a400f89085aef87e11467 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:53.837133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:53.837881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:53.837881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:53.857909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:53.859428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:53.926549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1127/2276] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:53.934699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:53.934993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:53.935357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:53.954777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:53.955554 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:53.965824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1150/2277] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:53.969951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:53.984310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ede54-71f7-4c52-9ec0-2e435e486883 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:53.992800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:53.993500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:53.993788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:53.994072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:53.996018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:29:53.996018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:53.997157 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.015127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1128/2278] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:29:54.016760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.026201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a412023-a0a5-468e-a7b5-32fdf4ef0e90 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.032576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.033407 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.033911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.034203 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.034931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.037599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.116192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1151/2279] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:54 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:54.120278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.130281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168078a-607d-47d4-870e-2c1e129ab501 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.196466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.197174 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.197462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.197744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.198474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.200560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.267744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:54.273948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1129/2280] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:54 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:54.284081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.289803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286daa63-3124-4302-b429-129a4ebd5ac8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.294578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.295264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.295591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.295865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.296583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.298219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.309818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1152/2281] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:54.313851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.324564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99aa86-74e3-4a12-9dc9-3dc155c51e00 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.329314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7f62f-3e69-4f8b-a05b-6b382dae3c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.329976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f62f-3e69-4f8b-a05b-6b382dae3c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.330262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f62f-3e69-4f8b-a05b-6b382dae3c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.330543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7f62f-3e69-4f8b-a05b-6b382dae3c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.331333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7f62f-3e69-4f8b-a05b-6b382dae3c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.332930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7f62f-3e69-4f8b-a05b-6b382dae3c0e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.344971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1130/2282] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:54.349331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.359889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961666-a321-413d-b3b5-a6725a82a1aa admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.364727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.365608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.365895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.366176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] PATH_INFO: `/v3/projects/7c26c3e029084baf95be400f04e78df9/users/b06ac7d95a92450a9320c7e8755968ce/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.381419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c26c3e029084baf95be400f04e78df9, user_id=b06ac7d95a92450a9320c7e8755968ce, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.384129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.483524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1153/2283] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:54 2022] PUT /identity/v3/projects/7c26c3e029084baf95be400f04e78df9/users/b06ac7d95a92450a9320c7e8755968ce/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:54.495340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.495643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.495927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.524783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] MFA Rules not processed for user `b06ac7d95a92450a9320c7e8755968ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:54.526126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:54.598050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1131/2284] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:54.602734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.680024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5f91-e46f-41b2-a98d-bcc0f6d272cf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.684619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.685292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.685562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.685837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.686531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.688478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.702108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:54.822861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1154/2285] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:54 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:54.824186 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:54.835471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c39da-af15-4ca0-b895-8623d0e25e67 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:54.894006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:54.894687 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:54.896657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:54.896657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:54.896657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:54.898587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:54.991821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1132/2286] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:54 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:29:54.996035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.007605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b519083-9aae-4f11-9b5c-89c5f2fa3711 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.012458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.013189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.013480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.013774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.014503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.016166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.028422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1155/2287] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:55.032757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.043602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc651250-3127-4e8e-bbd0-ee65a1c74b9f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.048279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358e7718-b2d2-402b-b94c-e068b363be81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.049111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e7718-b2d2-402b-b94c-e068b363be81 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.049397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e7718-b2d2-402b-b94c-e068b363be81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.049674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358e7718-b2d2-402b-b94c-e068b363be81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.050403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e7718-b2d2-402b-b94c-e068b363be81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.052045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358e7718-b2d2-402b-b94c-e068b363be81 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.062515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1133/2288] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:55.067040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.077665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a4d9d-d97b-46c2-9036-8ad73b98dab5 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.082589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.084082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.084366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.084640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] PATH_INFO: `/v3/projects/47c563dd990c45db80f5760b40ff67c4/users/eaeca2239eba40a7b08af4b087493df9/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.103738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c563dd990c45db80f5760b40ff67c4, user_id=eaeca2239eba40a7b08af4b087493df9, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.106512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.186352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1156/2289] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:55 2022] PUT /identity/v3/projects/47c563dd990c45db80f5760b40ff67c4/users/eaeca2239eba40a7b08af4b087493df9/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:29:55.198126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.198423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.198708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.229228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] MFA Rules not processed for user `eaeca2239eba40a7b08af4b087493df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:55.230484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:55.276784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:55.304667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1134/2290] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:55.309658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.371792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647a712-091e-4032-bdb4-56a67de447d2 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.376498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.377151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.377438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.377713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.378459 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.381112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.498366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1157/2291] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:29:55 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:55.502630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.514891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6018b7-00ac-424d-a14d-c9387b4d9a25 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.571998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.572676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.572952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.573224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.573930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.575869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.645348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1135/2292] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:29:55 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:29:55.649944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.660523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f3548-97e8-4a17-addd-9de3476f7a48 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.665542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.666277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.666558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.666835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.667618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.669316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.680903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1158/2293] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:55.686193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.696872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0ac9-3b7f-4895-84cc-c6e9a0611321 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.701913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.702795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.703075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.703404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] PATH_INFO: `/v3/projects/acdbb3070fb54f8685b4729996b2f265/users/42e8af71b6a3400796b20f8dd13bc65a/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.718800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdbb3070fb54f8685b4729996b2f265, user_id=42e8af71b6a3400796b20f8dd13bc65a, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.721569 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.795521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1136/2294] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:55 2022] PUT /identity/v3/projects/acdbb3070fb54f8685b4729996b2f265/users/42e8af71b6a3400796b20f8dd13bc65a/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:55.799619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.810391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b208d86-3c45-460b-bf5a-52a80a3bcdbf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.871678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c628236-e074-462e-87c7-095d0bd92430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.873081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628236-e074-462e-87c7-095d0bd92430 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.873081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628236-e074-462e-87c7-095d0bd92430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.873081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c628236-e074-462e-87c7-095d0bd92430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.874507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c628236-e074-462e-87c7-095d0bd92430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.875324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c628236-e074-462e-87c7-095d0bd92430 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.889352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1159/2295] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:29:55.893526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:55.906231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dafd2-d731-40ad-a427-64319c1888ef admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:55.912439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:55.913341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.913633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.913914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] PATH_INFO: `/v3/projects/acdbb3070fb54f8685b4729996b2f265/users/42e8af71b6a3400796b20f8dd13bc65a/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.915952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdbb3070fb54f8685b4729996b2f265, user_id=42e8af71b6a3400796b20f8dd13bc65a, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:55.919316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:55.941341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1137/2296] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:29:55 2022] PUT /identity/v3/projects/acdbb3070fb54f8685b4729996b2f265/users/42e8af71b6a3400796b20f8dd13bc65a/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:29:55.951697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:55.952088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:55.952440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:55.985526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] MFA Rules not processed for user `42e8af71b6a3400796b20f8dd13bc65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:29:55.986751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:56.004331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:56.071609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1160/2297] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:29:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:29:56.083983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:29:56.085974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:29:56.146511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60febb-6db1-442d-8d4b-86f8a725266a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:29:56.150142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:29:56.150792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:29:56.151075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:29:56.151389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:29:56.200247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:29:56.212760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:29:56.213337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:29:56.222273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1138/2298] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:29:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:01.379954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:01.391256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab861-d03c-4229-87f5-fc3d6622d3e3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:01.394802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:01.395524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:01.396297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:01.396575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:01.457007 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:01.458545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:01.468354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1161/2299] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:05.303365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:05.322980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad8f7d-b641-4b92-8a19-9b38bc2ed9fa service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:05.326745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759051c0-3839-4c80-b506-d474c67b6aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:05.327421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759051c0-3839-4c80-b506-d474c67b6aed service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:05.327710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759051c0-3839-4c80-b506-d474c67b6aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:05.327987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759051c0-3839-4c80-b506-d474c67b6aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:05.338412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759051c0-3839-4c80-b506-d474c67b6aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:05.340477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759051c0-3839-4c80-b506-d474c67b6aed service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:05.352785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1139/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:11.463980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=49 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:15.346105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=14 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:17.863060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:17.935753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ab60-9342-4c4d-bdcc-74b7df3d9ca0 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:17.940398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:17.941161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:17.941447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:17.941728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] PATH_INFO: `/v3/users/b06ac7d95a92450a9320c7e8755968ce` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:17.943179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06ac7d95a92450a9320c7e8755968ce)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:17.945805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:18.165442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06ac7d95a92450a9320c7e8755968ce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:18.167159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06ac7d95a92450a9320c7e8755968ce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:18.182733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06ac7d95a92450a9320c7e8755968ce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:18.193032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06ac7d95a92450a9320c7e8755968ce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:18.284441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06ac7d95a92450a9320c7e8755968ce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:18.292205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1162/2301] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:30:17 2022] DELETE /identity/v3/users/b06ac7d95a92450a9320c7e8755968ce => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:18.296819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:18.297288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:18.297398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:18.316277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:18.317501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:18.378058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1140/2302] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:18.388395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:18.389756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:18.453839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c827250-d15b-4a56-9816-fcec66a4905f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:18.457818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:18.458488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:18.458775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:18.459056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:18.471143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:18.472740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:18.484901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1163/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:18.952781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:18.954206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:18.968470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0701d12-a15b-4425-bf52-d2cb61844a75 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:18.973886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:18.974852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:18.975141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:18.975926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] PATH_INFO: `/v3/projects/7c26c3e029084baf95be400f04e78df9` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:18.976736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c26c3e029084baf95be400f04e78df9)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:18.979424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:19.131424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c26c3e029084baf95be400f04e78df9', 'request_id': 'req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.255874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c26c3e029084baf95be400f04e78df9', 'request_id': 'req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.262806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c26c3e029084baf95be400f04e78df9', 'request_id': 'req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.276416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] The token cache is being invalidate because project 7c26c3e029084baf95be400f04e78df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:30:19.276704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.277481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-66c6648e-99c5-4c1a-b9b9-13b11ae16aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.279195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1141/2304] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:30:18 2022] DELETE /identity/v3/projects/7c26c3e029084baf95be400f04e78df9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:19.283111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:19.293312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81017a6-61bf-46db-8dbe-75b1b7fe927d service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:19.312780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:19.352738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:19.353611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:19.353895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:19.354173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] PATH_INFO: `/v3/users/eaeca2239eba40a7b08af4b087493df9` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:19.355416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaeca2239eba40a7b08af4b087493df9)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:19.357608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:19.598194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7030419-cf57-4681-918a-5e180abda87b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:19.598686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7030419-cf57-4681-918a-5e180abda87b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:19.599185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7030419-cf57-4681-918a-5e180abda87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:19.620793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaeca2239eba40a7b08af4b087493df9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.621562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaeca2239eba40a7b08af4b087493df9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.640895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaeca2239eba40a7b08af4b087493df9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.641464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d7030419-cf57-4681-918a-5e180abda87b None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:19.642753 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d7030419-cf57-4681-918a-5e180abda87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:19.650108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaeca2239eba40a7b08af4b087493df9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.710627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1142/2305] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:30:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:30:19.751783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaeca2239eba40a7b08af4b087493df9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:19.754948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1164/2306] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:30:19 2022] DELETE /identity/v3/users/eaeca2239eba40a7b08af4b087493df9 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:19.770352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7030419-cf57-4681-918a-5e180abda87b None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:19.772811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d7030419-cf57-4681-918a-5e180abda87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:19.787680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:19.849094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7030419-cf57-4681-918a-5e180abda87b None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:19.909502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f906ad6-ac36-4c07-ad31-00e8fb005fae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:19.910161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f906ad6-ac36-4c07-ad31-00e8fb005fae service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:19.910451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f906ad6-ac36-4c07-ad31-00e8fb005fae service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:19.910731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f906ad6-ac36-4c07-ad31-00e8fb005fae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:19.970648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f906ad6-ac36-4c07-ad31-00e8fb005fae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:19.972334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f906ad6-ac36-4c07-ad31-00e8fb005fae service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:19.983360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1143/2307] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:30:20.361970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:20.363784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:20.441661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9f84d-0d13-49e5-baa7-f7ec4839ad84 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:20.447521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:20.448514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:20.448830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:20.449213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] PATH_INFO: `/v3/projects/47c563dd990c45db80f5760b40ff67c4` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:20.450471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47c563dd990c45db80f5760b40ff67c4)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:20.452486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:20.504041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:20.504246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:20.504403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:20.523106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] MFA Rules not processed for user `12816eca659542af9e41b92fdb674eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:20.524782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:20.581194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1144/2308] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:30:20.582258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:20.583576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:20.593290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:20.597422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47c563dd990c45db80f5760b40ff67c4', 'request_id': 'req-b3d9d3db-c229-40a3-86de-eb9136f48e76'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.598800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c961-0a4a-4be4-83f5-ac153abe51e6 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:20.603583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47c563dd990c45db80f5760b40ff67c4', 'request_id': 'req-b3d9d3db-c229-40a3-86de-eb9136f48e76'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.690723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:20.691676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:20.691960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:20.692244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] PATH_INFO: `/v3/users/2ea165e3847b4c728d81aec0c3a13619` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:20.693637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea165e3847b4c728d81aec0c3a13619)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:20.696405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:20.701932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47c563dd990c45db80f5760b40ff67c4', 'request_id': 'req-b3d9d3db-c229-40a3-86de-eb9136f48e76'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.721799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] The token cache is being invalidate because project 47c563dd990c45db80f5760b40ff67c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:30:20.722178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.722945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.727508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1165/2309] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:30:20 2022] DELETE /identity/v3/projects/47c563dd990c45db80f5760b40ff67c4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:20.730422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:20.731927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:20.754373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9d3db-c229-40a3-86de-eb9136f48e76 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:20.754373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:20.754373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:20.754373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:20.754956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] PATH_INFO: `/v3/users/42e8af71b6a3400796b20f8dd13bc65a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:20.755738 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e8af71b6a3400796b20f8dd13bc65a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:20.759810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:20.957427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea165e3847b4c728d81aec0c3a13619'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.965224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea165e3847b4c728d81aec0c3a13619'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.966017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea165e3847b4c728d81aec0c3a13619'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:20.978605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea165e3847b4c728d81aec0c3a13619'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.008780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e8af71b6a3400796b20f8dd13bc65a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.009415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e8af71b6a3400796b20f8dd13bc65a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.021683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e8af71b6a3400796b20f8dd13bc65a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.030189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e8af71b6a3400796b20f8dd13bc65a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.078694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-53884a16-63b4-4aac-a8c9-37150baad61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea165e3847b4c728d81aec0c3a13619'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.086097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1145/2310] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:30:20 2022] DELETE /identity/v3/users/2ea165e3847b4c728d81aec0c3a13619 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:21.088408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:21.088544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:21.088727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:21.109874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:21.145496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:21.147058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:21.154324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e2224ee2-cd3c-44ef-aa3a-cc4965909a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e8af71b6a3400796b20f8dd13bc65a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.168542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1166/2311] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:30:20 2022] DELETE /identity/v3/users/42e8af71b6a3400796b20f8dd13bc65a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:30:21.168542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:21.168542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:21.168542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:21.210367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:21.214992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:21.241847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1146/2312] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jun 1 04:30:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:21.255947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:21.257365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:21.292461 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1167/2313] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:21.302131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:21.303580 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:21.374393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66734abc-e424-48e9-91cb-76ce06376499 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:21.381112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819448fe-3610-4438-858b-16afe0fefd8f None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:21.386190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:21.386536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:21.386924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:21.387503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:21.397822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:21.400332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:21.411992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1168/2314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:30:21.460677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:21.461872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:21.462234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:21.462517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:21.488472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:21.556713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:21.556713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:21.565329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1147/2315] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:30:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 01 04:30:21.736651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:21.738183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:21.753515 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068ab7-1746-407c-9292-03dea615d550 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:21.762083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:21.768609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:21.768609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:21.768609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] PATH_INFO: `/v3/projects/acdbb3070fb54f8685b4729996b2f265` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:21.768609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acdbb3070fb54f8685b4729996b2f265)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:21.768609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:21.867847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:21.935749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acdbb3070fb54f8685b4729996b2f265', 'request_id': 'req-173f7221-ba7f-44d8-a9fb-d5361edd379f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:21.941338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acdbb3070fb54f8685b4729996b2f265', 'request_id': 'req-173f7221-ba7f-44d8-a9fb-d5361edd379f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.040142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.045646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.055769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588fad6-ec88-454b-a940-a00b2a7b49bd service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.072331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acdbb3070fb54f8685b4729996b2f265', 'request_id': 'req-173f7221-ba7f-44d8-a9fb-d5361edd379f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.089840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] The token cache is being invalidate because project acdbb3070fb54f8685b4729996b2f265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:30:22.090139 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.090807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-173f7221-ba7f-44d8-a9fb-d5361edd379f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.093399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1169/2316] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:30:21 2022] DELETE /identity/v3/projects/acdbb3070fb54f8685b4729996b2f265 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:22.098022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da82e57-df95-41f7-b1c1-43f682128b4f None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.098315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da82e57-df95-41f7-b1c1-43f682128b4f None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.098600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da82e57-df95-41f7-b1c1-43f682128b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.117525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-4da82e57-df95-41f7-b1c1-43f682128b4f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:22.119276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4da82e57-df95-41f7-b1c1-43f682128b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.127208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.128253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.128546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.128828 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] PATH_INFO: `/v3/projects/020f959e6c9942dda641d2488c175270` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.130056 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] RBAC: Authorizing `identity:delete_project(project_id=020f959e6c9942dda641d2488c175270)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.132052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.181655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1170/2317] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:22.189530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.189825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.190107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.210658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:22.212213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.226921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1171/2318] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:30:22.227874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.237233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93f2dd-cb61-4536-af53-0df461a09faa None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.242409 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.243746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.244658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.244949 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.245687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:30:22.246104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.247750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.260332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1172/2319] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:30:22.266385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.276294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2685d66-06d9-4723-8a3a-cd6730a88001 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.281231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.287034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.287034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.287034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.287034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.287034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.325101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020f959e6c9942dda641d2488c175270', 'request_id': 'req-173a4919-cf34-4313-8822-4e2f2e5f713e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.448662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1173/2320] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:30:22.452652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.473187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020f959e6c9942dda641d2488c175270', 'request_id': 'req-173a4919-cf34-4313-8822-4e2f2e5f713e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.482343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020f959e6c9942dda641d2488c175270', 'request_id': 'req-173a4919-cf34-4313-8822-4e2f2e5f713e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.498467 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] The token cache is being invalidate because project 020f959e6c9942dda641d2488c175270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:30:22.498761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.499646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-173a4919-cf34-4313-8822-4e2f2e5f713e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:22.501336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1148/2321] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:30:22 2022] DELETE /identity/v3/projects/020f959e6c9942dda641d2488c175270 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:30:22.505130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635166-6c93-4f77-ba13-ef76bfde2157 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.505590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635166-6c93-4f77-ba13-ef76bfde2157 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.505897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635166-6c93-4f77-ba13-ef76bfde2157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.520157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca388-ddb3-414e-9fd4-97f156b84a30 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.526436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a6635166-6c93-4f77-ba13-ef76bfde2157 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:22.527712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a6635166-6c93-4f77-ba13-ef76bfde2157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.532623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149af6-f72d-468e-a144-b793de340b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.533432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149af6-f72d-468e-a144-b793de340b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.533723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149af6-f72d-468e-a144-b793de340b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.534007 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149af6-f72d-468e-a144-b793de340b58 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.535257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6149af6-f72d-468e-a144-b793de340b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.536741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6149af6-f72d-468e-a144-b793de340b58 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.584599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1149/2322] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:22.585946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.586239 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.587130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.606065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:22.607870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.615935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1150/2323] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:22.616723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.618154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.668253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1174/2324] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:30:22.669566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.669883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.670196 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.689398 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:22.691539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.696721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf709d9a-9c7a-445c-8a08-940bfde6bc5e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.698968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.699742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.699950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.700232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.703781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1175/2325] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:22.704588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.706502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:22.709427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.711147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.719643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42294bfc-9392-4fe8-af4d-2fef0e55137d None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.720523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1151/2326] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:30:22.721403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.732359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb30022-9699-4c87-9f22-a537f3646a03 service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.732920 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.733600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.733888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.734173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.735723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.737561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.737707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.738456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.738739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.739021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.739794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:30:22.740132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.741762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.751332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1176/2327] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:30:22.753600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.757556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1152/2328] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:30:22.761168 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.764241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0e154-1925-4b79-9fb4-aab288d6991a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.769172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816db983-89ca-463f-8a3b-d2ba0483867b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.769329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.770049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.770406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.770751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.771678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.773293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.774656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.775323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.775636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.775918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.776605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.778531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.784708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1177/2329] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:30:22.788934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.798897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4dc6f-e894-4b94-b193-65b1cd6f3fbe admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.804788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f88d8c-5c2c-4772-a1fe-7f8ae576e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.807323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f88d8c-5c2c-4772-a1fe-7f8ae576e7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.807323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f88d8c-5c2c-4772-a1fe-7f8ae576e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.807323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f88d8c-5c2c-4772-a1fe-7f8ae576e7ff admin admin] PATH_INFO: `/v3/projects/572160e3226e4dd89400de4784bef70b/users/ab2124a7673e4b2bba48171274b85c50/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.824218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f88d8c-5c2c-4772-a1fe-7f8ae576e7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=572160e3226e4dd89400de4784bef70b, user_id=ab2124a7673e4b2bba48171274b85c50, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.826899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f88d8c-5c2c-4772-a1fe-7f8ae576e7ff admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:22.882920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1153/2330] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:30:22.885213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:22.947068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ae8fe-42ae-41b6-ac1a-9c8bc530ad5d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:22.951991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:22.952656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:22.952942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:22.953221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:22.953915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:22.955876 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.004722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1178/2331] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:30:22 2022] PUT /identity/v3/projects/572160e3226e4dd89400de4784bef70b/users/ab2124a7673e4b2bba48171274b85c50/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:30:23.024269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.024788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.025405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.054996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] MFA Rules not processed for user `ab2124a7673e4b2bba48171274b85c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:23.056730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:23.123242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1154/2332] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:30:22 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:30:23.125127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:23.129388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1179/2333] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:23.135981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809a3cb-a6d6-4c56-9eb7-d5ed011f81c2 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:23.142404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:23.143825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:23.168148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:23.204544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:23.205362 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.205665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.206048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.207042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:23.208847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.220097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1155/2334] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:30:23.224005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:23.228784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa35c0-342d-4302-a6c2-5909ec6ae635 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:23.234038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c81a-669f-4c9a-b04c-c4183e29fe69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:23.234719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c81a-669f-4c9a-b04c-c4183e29fe69 service glance] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.235078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c81a-669f-4c9a-b04c-c4183e29fe69 service glance] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.235487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c81a-669f-4c9a-b04c-c4183e29fe69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.236640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d8a84-d7dc-48f4-aa79-cf96b88905c9 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:23.242562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:23.243944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.244480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.244769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.245515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:23.247160 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.248393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c81a-669f-4c9a-b04c-c4183e29fe69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:23.250203 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c81a-669f-4c9a-b04c-c4183e29fe69 service glance] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.258912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1156/2335] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:30:23.264282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:23.266706 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1180/2336] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:30:23.275416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1312e-f45c-469c-b2c5-4fa89827e7a6 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:23.281080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:23.282065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.282909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.283556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] PATH_INFO: `/v3/projects/9369d530285849b3a9eac5e0034d3170/users/c052b2f518064843928b4eff569e913f/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.304724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9369d530285849b3a9eac5e0034d3170, user_id=c052b2f518064843928b4eff569e913f, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:23.311421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.441835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1157/2337] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:30:23 2022] PUT /identity/v3/projects/9369d530285849b3a9eac5e0034d3170/users/c052b2f518064843928b4eff569e913f/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:23.442287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe4c1-3b83-4197-8cac-d224098220d8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.442287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe4c1-3b83-4197-8cac-d224098220d8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.442287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe4c1-3b83-4197-8cac-d224098220d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.475587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-202fe4c1-3b83-4197-8cac-d224098220d8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:23.476886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-202fe4c1-3b83-4197-8cac-d224098220d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:23.490313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:23.547241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1181/2338] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:23.557524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:23.559497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:23.612065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.612783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.613419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.634794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:23.636348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:23.648360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:23.654332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adbcfe-80cf-41c7-a9eb-90e01583b9d1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:23.658245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:23.658908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.659196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.659516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.669824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:23.671671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.683583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1158/2339] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:23.698913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1182/2340] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jun 1 04:30:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:30:23.712089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:23.786523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425c6d8-bdde-4991-b64e-3c153213c738 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:23.792440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6f277-62d3-403e-b9a4-29b7fadfedbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:23.793127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f277-62d3-403e-b9a4-29b7fadfedbe service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:23.793411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f277-62d3-403e-b9a4-29b7fadfedbe service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:23.793686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6f277-62d3-403e-b9a4-29b7fadfedbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:23.802258 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6f277-62d3-403e-b9a4-29b7fadfedbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:23.804045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6f277-62d3-403e-b9a4-29b7fadfedbe service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:23.813271 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1159/2341] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:30:25.039801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:25.599827 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:27.298804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:27.300286 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:27.316912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4e647-95b7-421a-a3cc-7f555becc2fb None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:27.324346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:27.325555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:27.326140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:27.337873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] PATH_INFO: `/v3/users/be19da315701413e810d6e1274b36f2b` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:27.337873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] RBAC: Authorizing `identity:delete_user(user_id=be19da315701413e810d6e1274b36f2b)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:27.337873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:27.617456 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be19da315701413e810d6e1274b36f2b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:27.634847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be19da315701413e810d6e1274b36f2b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:27.672412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be19da315701413e810d6e1274b36f2b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:27.690272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be19da315701413e810d6e1274b36f2b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:27.715039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:27.777692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be19da315701413e810d6e1274b36f2b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:27.788276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1183/2342] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:30:27 2022] DELETE /identity/v3/users/be19da315701413e810d6e1274b36f2b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:27.789771 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:27.790270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:27.790390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:27.825394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:27.827148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:27.942414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1160/2343] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:30:27.952449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:27.954205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:28.035759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42123352-26a3-41ca-a715-a0cb6904206c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:28.058613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad24e96-7512-4335-8007-faa917bda593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:28.059332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad24e96-7512-4335-8007-faa917bda593 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:28.060189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad24e96-7512-4335-8007-faa917bda593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:28.060483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad24e96-7512-4335-8007-faa917bda593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:28.074616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad24e96-7512-4335-8007-faa917bda593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:28.076417 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad24e96-7512-4335-8007-faa917bda593 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:28.090800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1184/2344] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:28.499745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:28.501414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:28.513964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588c56e-a4e6-416c-b2f5-501e24b50b45 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:28.535851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:28.535851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:28.535851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:28.535851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] PATH_INFO: `/v3/projects/e2080f03ddd148889160f6f51aa71955` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:28.536641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2080f03ddd148889160f6f51aa71955)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:28.538865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:28.569666 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:28.696319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2080f03ddd148889160f6f51aa71955', 'request_id': 'req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:28.888800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2080f03ddd148889160f6f51aa71955', 'request_id': 'req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:28.903051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2080f03ddd148889160f6f51aa71955', 'request_id': 'req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:28.917793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] The token cache is being invalidate because project e2080f03ddd148889160f6f51aa71955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:30:28.918083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:28.919396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:28.920975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1161/2345] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:30:28 2022] DELETE /identity/v3/projects/e2080f03ddd148889160f6f51aa71955 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:30.728449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:30.728948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:30.729059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:30.781312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] MFA Rules not processed for user `c052b2f518064843928b4eff569e913f`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:30.782559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:30.883977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1185/2346] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:30.887014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:30.888412 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:30.993930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fc28c-6f08-492c-824f-6ebb7d5f0aa5 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:31.000008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:31.000706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:31.000993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:31.001270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:31.003055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:31.004257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:31.108262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1162/2347] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:30:30 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:30:31.111506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:31.131751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ec1d1-aeaf-449a-ab84-7a438d237b5b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:31.212525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:31.213288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:31.213578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:31.213999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:31.214921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:31.217149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:31.323686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1186/2348] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:30:31 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:30:31.328220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:31.346868 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35f949-4813-4bd4-831f-0b2d180d8817 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:31.354974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:31.356167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:31.356167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:31.356488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:31.357556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:31.359817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:31.382277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:31.385554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1163/2349] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:30:31.397055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bfc71-7342-4f55-8555-ecfb38dbfbd3 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:31.405003 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:31.405908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:31.406190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:31.406465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] PATH_INFO: `/v3/projects/f6280619df0e4dac84868ea94e7885e8/users/f7289e7b11b6428b8295a05748ebd882/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:31.427973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6280619df0e4dac84868ea94e7885e8, user_id=f7289e7b11b6428b8295a05748ebd882, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:31.430925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:31.433959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:31.514677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1187/2350] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:30:31 2022] PUT /identity/v3/projects/f6280619df0e4dac84868ea94e7885e8/users/f7289e7b11b6428b8295a05748ebd882/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:31.518397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:31.530618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e089-8d8c-46e2-bfde-e6d95961c91d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:31.612635 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:31.613413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:31.613859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:31.614257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:31.615734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:31.618716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:31.882877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1164/2351] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:30:31.886924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:31.886924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a3e03-12e8-466d-ac86-582bfdbbf761 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:31.886924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14fe26-4e18-40c9-90d5-a6e973b20521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:31.886924 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14fe26-4e18-40c9-90d5-a6e973b20521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:31.889073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14fe26-4e18-40c9-90d5-a6e973b20521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:31.889073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14fe26-4e18-40c9-90d5-a6e973b20521 admin admin] PATH_INFO: `/v3/projects/f6280619df0e4dac84868ea94e7885e8/users/f7289e7b11b6428b8295a05748ebd882/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:31.889073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14fe26-4e18-40c9-90d5-a6e973b20521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6280619df0e4dac84868ea94e7885e8, user_id=f7289e7b11b6428b8295a05748ebd882, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:31.889073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14fe26-4e18-40c9-90d5-a6e973b20521 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:31.889073 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1188/2352] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:30:31 2022] PUT /identity/v3/projects/f6280619df0e4dac84868ea94e7885e8/users/f7289e7b11b6428b8295a05748ebd882/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:39.405530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:39.405530 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:39.475713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:39.728043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1c84-ae35-4e0b-a131-0164fbeb8e36 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:39.861097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:39.869922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:39.869922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:39.869922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:39.869922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:39.879982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:39.883235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:39.895867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1165/2353] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:40.617994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:40.617994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:40.617994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:40.665521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] MFA Rules not processed for user `f7289e7b11b6428b8295a05748ebd882`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:40.666942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:40.752008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1189/2354] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:40.773553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:40.775937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:40.903122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b48f6-f24d-491c-8c20-8f4c6efa47e4 service placement] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:40.908960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdd327-4b41-473d-a36a-7aca69bf4953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:40.909619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdd327-4b41-473d-a36a-7aca69bf4953 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:40.910401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdd327-4b41-473d-a36a-7aca69bf4953 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:40.910401 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdd327-4b41-473d-a36a-7aca69bf4953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:40.918977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:40.983497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdd327-4b41-473d-a36a-7aca69bf4953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:40.988040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdd327-4b41-473d-a36a-7aca69bf4953 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:40.996222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1166/2355] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:43.107978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:43.111761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:43.449337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f828c5-eb4f-43de-82e8-5b3152178ad4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:43.455945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:43.456856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:43.457145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:43.457431 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] PATH_INFO: `/v3/users/e4a9e14a3aff4da1a2e31d4f8948dd59` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:43.458686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a9e14a3aff4da1a2e31d4f8948dd59)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:43.460833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:43.750305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a9e14a3aff4da1a2e31d4f8948dd59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:43.751715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a9e14a3aff4da1a2e31d4f8948dd59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:43.777448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a9e14a3aff4da1a2e31d4f8948dd59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:43.780119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:43.786134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a9e14a3aff4da1a2e31d4f8948dd59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:43.890830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a9e14a3aff4da1a2e31d4f8948dd59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:43.905531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1190/2356] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:30:43 2022] DELETE /identity/v3/users/e4a9e14a3aff4da1a2e31d4f8948dd59 => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:43.906325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:43.906786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:43.906890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:43.939052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:43.940589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:44.028894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1167/2357] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:30:44.043130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:44.044794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:44.117241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b93b81-bb46-44fb-af3f-41a7891e42f4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:44.121433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b125509-08b2-4e76-a147-6eb2c3e40d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:44.122087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b125509-08b2-4e76-a147-6eb2c3e40d60 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:44.122371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b125509-08b2-4e76-a147-6eb2c3e40d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:44.122653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b125509-08b2-4e76-a147-6eb2c3e40d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:44.133247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b125509-08b2-4e76-a147-6eb2c3e40d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:44.134641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b125509-08b2-4e76-a147-6eb2c3e40d60 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:44.146255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1191/2358] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:44.647396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:44.650439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:44.671904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39c1e-3012-4a51-8bff-236c5f793ca4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:44.684862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:44.685913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:44.686391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:44.687020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] PATH_INFO: `/v3/projects/9f84c80ebf3c492baf85c4720ade1090` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:44.688787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f84c80ebf3c492baf85c4720ade1090)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:44.693954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:44.882421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f84c80ebf3c492baf85c4720ade1090', 'request_id': 'req-c872be47-f63c-462b-b3b5-69df61a8a05a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:44.981075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f84c80ebf3c492baf85c4720ade1090', 'request_id': 'req-c872be47-f63c-462b-b3b5-69df61a8a05a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:44.987915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f84c80ebf3c492baf85c4720ade1090', 'request_id': 'req-c872be47-f63c-462b-b3b5-69df61a8a05a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:45.009849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] The token cache is being invalidate because project 9f84c80ebf3c492baf85c4720ade1090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:30:45.010145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:45.010818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c872be47-f63c-462b-b3b5-69df61a8a05a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:30:45.017372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.017669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.018007 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.020810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1168/2359] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:30:44 2022] DELETE /identity/v3/projects/9f84c80ebf3c492baf85c4720ade1090 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:45.053231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:45.054608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:45.180664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1192/2360] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:45.187146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.187956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.188238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.228611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:45.228611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:45.253935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1169/2361] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:45.257772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:45.270410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f66848-6ae7-4a84-8346-868fa5743557 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:45.280363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:45.280363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.280363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.280363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.280895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:30:45.280895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:45.280895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:45.307039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1193/2362] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:30:45.311097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:45.329918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9e7b-92c5-44d0-919b-eae7c2385ed9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:45.342705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:45.343903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.344374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.344826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.345843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:45.349671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:45.513440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1170/2363] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:30:45 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:30:45.518090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:45.536677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f320018-678a-475c-a7d7-52d55531c8e7 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:45.615059 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:45.615995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.618046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.618815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.619857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:45.622113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:45.736696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1194/2364] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:30:45 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:30:45.740864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:45.746890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:45.752177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a7cce-845c-44cd-b463-625710a3dd4f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:45.756707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:45.762861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.762861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.762861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.762861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:45.762861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:45.771509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1171/2365] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:30:45.775811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:45.782782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:45.788118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473151fa-b068-4aff-a587-6fedd07eb6f0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:45.793481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5e902-5709-4932-a139-3c5a759963cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:45.794150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5e902-5709-4932-a139-3c5a759963cc admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.797419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5e902-5709-4932-a139-3c5a759963cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.797419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5e902-5709-4932-a139-3c5a759963cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.797419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5e902-5709-4932-a139-3c5a759963cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:45.797419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5e902-5709-4932-a139-3c5a759963cc admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:45.809129 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1195/2366] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:30:45.813221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:45.823514 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccff006-b647-4da5-a241-ff0a6cb551e6 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:45.828474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:45.829358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:45.829640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:45.829917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] PATH_INFO: `/v3/projects/693843a5d3c04e3a9ff65ac8959d16d7/users/9075d53ca48d408a947b85ff053d93a9/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:45.845411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693843a5d3c04e3a9ff65ac8959d16d7, user_id=9075d53ca48d408a947b85ff053d93a9, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:45.848102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:45.999731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1172/2367] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:30:45 2022] PUT /identity/v3/projects/693843a5d3c04e3a9ff65ac8959d16d7/users/9075d53ca48d408a947b85ff053d93a9/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:30:46.011168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:46.011482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:46.011764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:46.042160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] MFA Rules not processed for user `9075d53ca48d408a947b85ff053d93a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:30:46.042863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:46.124191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1196/2368] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:30:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:30:46.135998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:30:46.138035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:30:46.209977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e55bff-6ba3-4efe-8dbf-d00ece765f28 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:30:46.215234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9c2e-a00b-472f-a172-358717076601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:30:46.216248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9c2e-a00b-472f-a172-358717076601 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:30:46.216248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9c2e-a00b-472f-a172-358717076601 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:30:46.216573 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9c2e-a00b-472f-a172-358717076601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:30:46.228383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9c2e-a00b-472f-a172-358717076601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:30:46.230054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff9c2e-a00b-472f-a172-358717076601 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:30:46.246583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1173/2369] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:30:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:30:56.833367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=45,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:30:56.959317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=35 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:11.776795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:11.839112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a718-52ab-4bee-bc47-294f0da8c1cf None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:11.844209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:11.845083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:11.845367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:11.845642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] PATH_INFO: `/v3/users/9075d53ca48d408a947b85ff053d93a9` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:11.846822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9075d53ca48d408a947b85ff053d93a9)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:11.848789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:12.031673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9075d53ca48d408a947b85ff053d93a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.032478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9075d53ca48d408a947b85ff053d93a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.045742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9075d53ca48d408a947b85ff053d93a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.055329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9075d53ca48d408a947b85ff053d93a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.121502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9075d53ca48d408a947b85ff053d93a9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.129452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1197/2370] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:31:11 2022] DELETE /identity/v3/users/9075d53ca48d408a947b85ff053d93a9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:31:12.134872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:12.135396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:12.135973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:12.153892 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:12.155717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:12.220568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1174/2371] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:12.227198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:12.228748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:12.297754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cf5ee-02aa-414d-bc7b-a9c13dc06ae5 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:12.301445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2c25a-1f36-40ff-aa15-a058f1cf5c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:12.302164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c25a-1f36-40ff-aa15-a058f1cf5c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:12.302394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c25a-1f36-40ff-aa15-a058f1cf5c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:12.302673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c25a-1f36-40ff-aa15-a058f1cf5c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:12.313025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2c25a-1f36-40ff-aa15-a058f1cf5c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:12.314485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2c25a-1f36-40ff-aa15-a058f1cf5c6e service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:12.324518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1198/2372] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:31:12.775116 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:12.776592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:12.789488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a3b52-aac6-487c-b11e-9c316cd008f2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:12.796768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:12.800926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:12.800926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:12.800926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] PATH_INFO: `/v3/projects/693843a5d3c04e3a9ff65ac8959d16d7` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:12.806765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=693843a5d3c04e3a9ff65ac8959d16d7)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:12.809155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:12.924714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693843a5d3c04e3a9ff65ac8959d16d7', 'request_id': 'req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.992486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693843a5d3c04e3a9ff65ac8959d16d7', 'request_id': 'req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:12.998973 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693843a5d3c04e3a9ff65ac8959d16d7', 'request_id': 'req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:13.010483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] The token cache is being invalidate because project 693843a5d3c04e3a9ff65ac8959d16d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:31:13.010775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:13.011485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:13.013034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1175/2373] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:31:12 2022] DELETE /identity/v3/projects/693843a5d3c04e3a9ff65ac8959d16d7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:31:21.813418 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9195b-fb60-4826-a1ae-ed45462e34f2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:21.814079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9195b-fb60-4826-a1ae-ed45462e34f2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:21.814079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9195b-fb60-4826-a1ae-ed45462e34f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:21.831973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-84c9195b-fb60-4826-a1ae-ed45462e34f2 None None] MFA Rules not processed for user `c052b2f518064843928b4eff569e913f`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:21.833435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-84c9195b-fb60-4826-a1ae-ed45462e34f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:21.851150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:21.888387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1199/2374] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:23.007423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:24.981466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:24.983050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:25.051787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4faff4-58e3-4fcb-a67e-e18b78928fc6 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:25.056485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:25.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:25.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:25.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] PATH_INFO: `/v3/users/ab2124a7673e4b2bba48171274b85c50` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:25.060811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2124a7673e4b2bba48171274b85c50)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:25.061129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:25.221598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2124a7673e4b2bba48171274b85c50'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:25.229142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2124a7673e4b2bba48171274b85c50'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:25.229854 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2124a7673e4b2bba48171274b85c50'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:25.248562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2124a7673e4b2bba48171274b85c50'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:25.304329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2124a7673e4b2bba48171274b85c50'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:25.310674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1176/2375] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:31:24 2022] DELETE /identity/v3/users/ab2124a7673e4b2bba48171274b85c50 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:31:25.319496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:25.319496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:25.319496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:25.335405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:25.336609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:25.429216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1200/2376] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:31:25.444188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:25.445641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:25.503679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96891dc6-8a93-4d18-ad88-3e44b292eb8e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:25.507252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88043cfe-8d34-460f-9bda-964634393270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:25.507928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88043cfe-8d34-460f-9bda-964634393270 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:25.508218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88043cfe-8d34-460f-9bda-964634393270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:25.508500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88043cfe-8d34-460f-9bda-964634393270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:25.518909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88043cfe-8d34-460f-9bda-964634393270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:25.520379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88043cfe-8d34-460f-9bda-964634393270 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:25.529931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1177/2377] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:31:25.982212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:25.983718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:26.002070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6615f-bba5-4e76-b08c-12959e62838a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.008655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.009664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.009955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.010233 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] PATH_INFO: `/v3/projects/572160e3226e4dd89400de4784bef70b` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.012438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572160e3226e4dd89400de4784bef70b)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.015355 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.186094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572160e3226e4dd89400de4784bef70b', 'request_id': 'req-109fbab3-a44b-4bed-b865-2c6082444f04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:26.195917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572160e3226e4dd89400de4784bef70b', 'request_id': 'req-109fbab3-a44b-4bed-b865-2c6082444f04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:26.254480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572160e3226e4dd89400de4784bef70b', 'request_id': 'req-109fbab3-a44b-4bed-b865-2c6082444f04'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:26.286883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] The token cache is being invalidate because project 572160e3226e4dd89400de4784bef70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:31:26.287178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:26.288056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-109fbab3-a44b-4bed-b865-2c6082444f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:31:26.289894 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1201/2378] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:31:25 2022] DELETE /identity/v3/projects/572160e3226e4dd89400de4784bef70b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:31:26.294792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.295354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.295705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.326964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:26.332673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:26.365652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:26.411576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1178/2379] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:26.420654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.421150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.425428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.442173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:26.443571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:26.451774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1202/2380] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:31:26.456516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:26.466549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4c193-1ec9-4fb7-b133-162bc440ca88 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.472021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.472696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.475760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.475760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.475760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:31:26.475760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.476711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.493944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1179/2381] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:31:26.495288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:26.505983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc092388-3fd5-4360-9a2d-59d973b06ba4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.510895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.511697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.511987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.512267 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.515669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.515669 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.603590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1203/2382] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:31:26 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:31:26.607839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:26.618266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb24e-d6c1-4b4e-a481-e750972e105a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.688824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.689579 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.689928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.690292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.691059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.693102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.787554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1180/2383] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:31:26 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:31:26.787813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:26.803537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaff9d8-83c7-44ce-9341-9e00ebea3b59 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.814447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.815832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.815832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.815832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.816486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.818128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.832179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1204/2384] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:31:26.835259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:26.846209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdded285-6db6-40cc-b093-bae7c571980a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.851431 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372151-8c65-426a-ab79-3a8e7fdac695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.852207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372151-8c65-426a-ab79-3a8e7fdac695 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.852565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372151-8c65-426a-ab79-3a8e7fdac695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.852921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372151-8c65-426a-ab79-3a8e7fdac695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.853814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372151-8c65-426a-ab79-3a8e7fdac695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.855600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372151-8c65-426a-ab79-3a8e7fdac695 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.867398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1181/2385] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:31:26.871387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:26.882149 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763cb63-ea62-4f7c-950d-0c687bb18e39 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:26.887123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:26.888133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.888579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.889010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] PATH_INFO: `/v3/projects/7454aaeeba6b4a8c86a0446ae68dd4f1/users/e249c6bada9d4dcb91b83a0a29ad552d/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:26.901951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:26.908056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7454aaeeba6b4a8c86a0446ae68dd4f1, user_id=e249c6bada9d4dcb91b83a0a29ad552d, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:26.911217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:26.974385 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1205/2386] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:31:26 2022] PUT /identity/v3/projects/7454aaeeba6b4a8c86a0446ae68dd4f1/users/e249c6bada9d4dcb91b83a0a29ad552d/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:31:26.979983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd5b0-6d4f-4940-8cdf-5eac2c45d990 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:26.980481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd5b0-6d4f-4940-8cdf-5eac2c45d990 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:26.981317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd5b0-6d4f-4940-8cdf-5eac2c45d990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:27.000972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-acdbd5b0-6d4f-4940-8cdf-5eac2c45d990 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:27.004959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-acdbd5b0-6d4f-4940-8cdf-5eac2c45d990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:27.078763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1182/2387] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:27.079159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:27.079159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:27.142976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0491eab-8787-4665-ada3-0fb3a0aad34f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:27.145989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:27.146794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:27.147148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:27.147529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:27.161433 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:27.163605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:27.177157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1206/2388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:31:29.056768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d71d-9f9f-42f2-99e8-3757e0920b89 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:29.057307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d71d-9f9f-42f2-99e8-3757e0920b89 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:29.057744 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926d71d-9f9f-42f2-99e8-3757e0920b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:29.093511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5926d71d-9f9f-42f2-99e8-3757e0920b89 None None] MFA Rules not processed for user `e249c6bada9d4dcb91b83a0a29ad552d`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:29.095897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5926d71d-9f9f-42f2-99e8-3757e0920b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:29.124426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:29.183576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1183/2389] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:29.192131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:29.193817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:29.258892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0806f9-9fc7-45e4-aba4-ce8024d41b56 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:29.262614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c30bd-5f24-4b5d-b6d0-7b73c5a8cfd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:29.263289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c30bd-5f24-4b5d-b6d0-7b73c5a8cfd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:29.263608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c30bd-5f24-4b5d-b6d0-7b73c5a8cfd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:29.263888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c30bd-5f24-4b5d-b6d0-7b73c5a8cfd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:29.273108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c30bd-5f24-4b5d-b6d0-7b73c5a8cfd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:29.274556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c30bd-5f24-4b5d-b6d0-7b73c5a8cfd9 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:29.284467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1207/2390] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:31:31.947143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:31.947660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:31.947749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:31.966445 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] MFA Rules not processed for user `e249c6bada9d4dcb91b83a0a29ad552d`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:31.967678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:31.975165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1184/2391] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:31:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:37.099518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:37.100087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:37.100192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:37.129416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:37.130863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:37.202898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1208/2392] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:31:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:31:37.349900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:37.351509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:37.435601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ea3de-3bee-49fa-b740-192713095676 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:37.442636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef500623-22a7-4683-90bf-6e49ea71e996 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:37.443394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef500623-22a7-4683-90bf-6e49ea71e996 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:37.443644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef500623-22a7-4683-90bf-6e49ea71e996 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:37.443927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef500623-22a7-4683-90bf-6e49ea71e996 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:37.453147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef500623-22a7-4683-90bf-6e49ea71e996 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:37.456080 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef500623-22a7-4683-90bf-6e49ea71e996 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:37.464939 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1185/2393] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:31:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:31:46.163564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:46.167692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:46.188420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a49ce-3b91-49d6-9f5e-a725369a11d2 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:46.198154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:46.198829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:46.199116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:46.199424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:46.201329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:46.281629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:46.334739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:46.336530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35138f-bf5b-4083-93cc-8dccab76f7da service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:46.352152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1209/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:31:47.463320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=54 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:31:55.388822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698a9a5-d94a-4a2c-bc6c-7c5f6c89bed4 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:55.394778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698a9a5-d94a-4a2c-bc6c-7c5f6c89bed4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:55.394778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698a9a5-d94a-4a2c-bc6c-7c5f6c89bed4 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:55.394778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1186/2395] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:31:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 01 04:31:55.398822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce491e7-481c-4b2a-a241-b7b98bcf3b6d None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:55.398822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce491e7-481c-4b2a-a241-b7b98bcf3b6d None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:55.399289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce491e7-481c-4b2a-a241-b7b98bcf3b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:55.453662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5ce491e7-481c-4b2a-a241-b7b98bcf3b6d None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:55.455127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5ce491e7-481c-4b2a-a241-b7b98bcf3b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:55.528536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1210/2396] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:31:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:55.540804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698a9a5-d94a-4a2c-bc6c-7c5f6c89bed4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:55.542419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3698a9a5-d94a-4a2c-bc6c-7c5f6c89bed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:55.599824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698a9a5-d94a-4a2c-bc6c-7c5f6c89bed4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:55.603392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacbb28-c831-48a4-9336-5a1f7888b30a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:55.604045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacbb28-c831-48a4-9336-5a1f7888b30a service glance] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:55.604335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacbb28-c831-48a4-9336-5a1f7888b30a service glance] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:55.604619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacbb28-c831-48a4-9336-5a1f7888b30a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:55.615630 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacbb28-c831-48a4-9336-5a1f7888b30a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:55.617063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacbb28-c831-48a4-9336-5a1f7888b30a service glance] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:55.628143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1187/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:31:55.833878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef0841-0f9b-4976-85ed-74f4b0be0f50 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:55.834097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef0841-0f9b-4976-85ed-74f4b0be0f50 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:55.834383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef0841-0f9b-4976-85ed-74f4b0be0f50 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:55.835832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1211/2398] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:31:55.840642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d40f7-98ca-41bd-972e-0a76fd5fde4b None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:55.840959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d40f7-98ca-41bd-972e-0a76fd5fde4b None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:55.841295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d40f7-98ca-41bd-972e-0a76fd5fde4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:55.860223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c18d40f7-98ca-41bd-972e-0a76fd5fde4b None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:55.861410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c18d40f7-98ca-41bd-972e-0a76fd5fde4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:55.869657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1188/2399] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:31:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:55.883132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef0841-0f9b-4976-85ed-74f4b0be0f50 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:55.895751 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef0841-0f9b-4976-85ed-74f4b0be0f50 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:55.899292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5665a4-a2f2-42b0-b29d-179f0d3b8cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:55.899943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5665a4-a2f2-42b0-b29d-179f0d3b8cbf service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:55.900226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5665a4-a2f2-42b0-b29d-179f0d3b8cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:55.900502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5665a4-a2f2-42b0-b29d-179f0d3b8cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:55.909685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5665a4-a2f2-42b0-b29d-179f0d3b8cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:55.911121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5665a4-a2f2-42b0-b29d-179f0d3b8cbf service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:55.919053 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1212/2400] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:31:56.011886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b91f-cb19-4f91-9bf9-0fc8dd415237 None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:56.012201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b91f-cb19-4f91-9bf9-0fc8dd415237 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:56.012678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b91f-cb19-4f91-9bf9-0fc8dd415237 None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:56.014099 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1189/2401] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:31:56.019542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d44e-705a-4d12-a3db-01b5329d642b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:56.019836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d44e-705a-4d12-a3db-01b5329d642b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:56.020108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d44e-705a-4d12-a3db-01b5329d642b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:56.040160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5719d44e-705a-4d12-a3db-01b5329d642b None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:31:56.041393 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5719d44e-705a-4d12-a3db-01b5329d642b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:31:56.051573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1213/2402] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:31:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:31:56.062133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b91f-cb19-4f91-9bf9-0fc8dd415237 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:31:56.073432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b91f-cb19-4f91-9bf9-0fc8dd415237 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:31:56.077256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:31:56.077902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:31:56.078188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:31:56.078466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:31:56.093111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:31:56.093111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:31:56.102924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1190/2403] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:31:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:32:00.576481 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:00.576938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:00.577086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:00.597167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:32:00.598426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:00.606520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1214/2404] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:32:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:32:00.620253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:00.632458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670f77-392a-4909-968a-2044056fe986 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:00.637451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:00.638177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:00.638465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:00.638748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:00.647593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:00.649187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:00.657143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1191/2405] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:32:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:32:10.590491 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=62 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:32:10.653999 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=41 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:32:29.221069 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:29.230929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55511d-7420-436e-8b82-f5f2ad754a1b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:29.234863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:29.235634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:29.235910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:29.236185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:29.238246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:29.252317 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:29.253735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46564e8e-f16e-49eb-8d01-fac2ebc7d63a service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:29.264475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1215/2406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:32:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:32:31.611388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:31.623335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d7e87-c80a-45ab-9a9f-2ec6d61e7a91 service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:31.626322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:31.626970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:31.627864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:31.628145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:31.629932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:31.643863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:31.645288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:31.656504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1192/2407] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:32:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:32:37.559767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:37.560254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:37.560349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:37.580330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:32:37.581699 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:37.589485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1216/2408] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jun 1 04:32:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:32:37.613516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:37.626069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104264a-939d-4ee3-b31f-b712561eaffa service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:37.631614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:37.632345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:37.632789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:37.633026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:37.642171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:37.643842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:37.661337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1193/2409] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:32:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:32:38.667977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:38.669329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:38.685477 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0e10b-b62e-47d0-8c2d-ccf4a78c9858 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:38.690977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:38.691659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:38.691948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:38.692276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:38.694712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:38.753886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:38.755526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:38.765916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1217/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:32:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:32:44.239531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:44.249268 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341d49-42bf-4033-ae16-0fa917ad9b18 service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:44.252946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:44.253604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:44.253884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:44.257185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:44.257185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:44.268275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:44.269726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44065b8a-f872-4b5b-a473-5a9226569f50 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:44.278884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1194/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:32:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:32:44.484081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:32:44.496310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6071bb4-7456-4fbd-9301-b8bcaef220cb service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:32:44.502229 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:32:44.502885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:32:44.503175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:32:44.503498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:32:44.505337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:32:44.568058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:32:44.569661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:32:44.580006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1218/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:32:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:32:54.274899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=18 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:32:54.575596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=65 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:33:15.396689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:33:15.397202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:33:15.397301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:33:15.416946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] MFA Rules not processed for user `e249c6bada9d4dcb91b83a0a29ad552d`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:33:15.418736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:15.426032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1195/2413] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:33:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:33:15.659679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:33:15.670240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7a08b-a06f-461d-bb27-ae4cea113257 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:33:15.673828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:33:15.674538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:33:15.674821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:33:15.675096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:33:15.676890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:15.686400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:33:15.687902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:33:15.696074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1219/2414] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:33:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:33:16.124182 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:33:16.125799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:16.220100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591333a-1b41-41a7-b3ab-854c8df6857c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:33:16.226898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:33:16.227644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:33:16.227935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:33:16.228219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:33:16.230228 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:16.246565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:33:16.248161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707d271-da6a-4d73-bd21-135318b7dcb1 service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:33:16.258480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1196/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:33:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:33:20.827551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:33:20.829362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:20.847806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02570c4b-017f-4cc3-8ab1-959c859c5466 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:33:20.853410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:33:20.854381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:33:20.854381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:33:20.854663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:33:20.857043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:20.869808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:33:20.871457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:33:20.882670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1220/2416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:33:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:33:24.462714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da63e3-1e98-4b4d-a185-2f876ae22ce0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:33:24.463340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da63e3-1e98-4b4d-a185-2f876ae22ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:33:24.463340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da63e3-1e98-4b4d-a185-2f876ae22ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:33:24.480770 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-30da63e3-1e98-4b4d-a185-2f876ae22ce0 None None] MFA Rules not processed for user `12816eca659542af9e41b92fdb674eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:33:24.481972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-30da63e3-1e98-4b4d-a185-2f876ae22ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:24.489111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1197/2417] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:33:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:33:30.878667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=12 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:33:31.066579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:33:31.078408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453ce1d-a808-4499-b67f-742cd23dad14 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:33:31.083901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:33:31.084620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:33:31.084913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:33:31.085193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:33:31.086994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:33:31.099359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:33:31.100936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:33:31.112188 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1221/2418] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:33:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:33:34.475581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=38 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:33:41.106851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=6 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:26.196177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:26.197762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:26.197762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:26.220789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] MFA Rules not processed for user `85df2f6595d3489bb5d1d547cb2cd900`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:26.225161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:26.228886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1198/2419] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:34:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:36.214495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:39.025301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:39.037252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11997a-5a1e-49d1-ab38-f9e6ae1e7832 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:39.042450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:39.043087 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:39.043475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:39.043758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:39.045517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:39.058944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:39.060789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:39.072724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1222/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:34:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:34:39.206761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:39.218605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b185f3-8fb4-4c2d-87ac-634db217e46e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:39.224663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:39.224663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:39.224663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:39.224663 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] PATH_INFO: `/v3/users/e249c6bada9d4dcb91b83a0a29ad552d` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:39.225714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e249c6bada9d4dcb91b83a0a29ad552d)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:39.227679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:39.406026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e249c6bada9d4dcb91b83a0a29ad552d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.413072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e249c6bada9d4dcb91b83a0a29ad552d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.413760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e249c6bada9d4dcb91b83a0a29ad552d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.426222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e249c6bada9d4dcb91b83a0a29ad552d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.496864 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a1d2a8b7-a6e4-494b-a84b-a4580dfe820e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e249c6bada9d4dcb91b83a0a29ad552d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.504008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1199/2421] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:34:39 2022] DELETE /identity/v3/users/e249c6bada9d4dcb91b83a0a29ad552d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:34:39.754911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:39.765896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42506eaa-721c-4580-a083-b9505fe516f3 service placement] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:39.823552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:39.824414 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:39.826728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:39.826728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] PATH_INFO: `/v3/projects/7454aaeeba6b4a8c86a0446ae68dd4f1` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:39.826728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7454aaeeba6b4a8c86a0446ae68dd4f1)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:39.828146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:39.926523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7454aaeeba6b4a8c86a0446ae68dd4f1', 'request_id': 'req-0eb5e721-e1d8-4b00-8495-d8b399672554'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.932871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7454aaeeba6b4a8c86a0446ae68dd4f1', 'request_id': 'req-0eb5e721-e1d8-4b00-8495-d8b399672554'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:39.994168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7454aaeeba6b4a8c86a0446ae68dd4f1', 'request_id': 'req-0eb5e721-e1d8-4b00-8495-d8b399672554'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:40.006090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] The token cache is being invalidate because project 7454aaeeba6b4a8c86a0446ae68dd4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:34:40.006379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:40.007008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-0eb5e721-e1d8-4b00-8495-d8b399672554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:34:40.008540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1223/2422] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:34:39 2022] DELETE /identity/v3/projects/7454aaeeba6b4a8c86a0446ae68dd4f1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:34:40.013637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.014022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.014310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.033278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:40.034487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:40.094534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1200/2423] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:40.102492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.102986 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.103091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.121695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:40.122902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:40.131026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1224/2424] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:40.135224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.145312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3805dc-77b5-4556-a821-5e1c04c29237 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.150161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.150824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.151114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.151432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.152173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:34:40.152486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.154086 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.166129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1201/2425] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:34:40.170342 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.180301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da26e2-ef7c-401f-890b-df6921a923e8 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.185845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.186498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.186789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.187066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.187803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.189712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.294757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1225/2426] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:34:40 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:34:40.299201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.310201 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6d34-c3ec-4c1d-8825-ca2998ecb1aa admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.371185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.371915 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.372319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.372657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.373356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.375933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.382791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:40.469589 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1202/2427] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:34:40 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:34:40.472201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.482632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31424402-db71-4674-a5d9-6f072b26ccbd admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.488737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.489471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.489821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.490177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.490972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.492904 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.503778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1226/2428] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:34:40.507722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.518447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8ef0-1c96-4a26-8551-5e38ab7fa1d7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.523577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f19f2-96b4-449b-8d8f-24f73ccc2e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.524489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f19f2-96b4-449b-8d8f-24f73ccc2e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.525166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f19f2-96b4-449b-8d8f-24f73ccc2e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.525684 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f19f2-96b4-449b-8d8f-24f73ccc2e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.526414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f19f2-96b4-449b-8d8f-24f73ccc2e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.528098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f19f2-96b4-449b-8d8f-24f73ccc2e5a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.539581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1203/2429] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:34:40.543613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.554444 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3f72e-9930-4b6e-ba68-a965b761faad admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.559550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.561281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.561281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.561480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] PATH_INFO: `/v3/projects/cd5966396128491d814472e980cfd873/users/43b1ece3cb2e4fd6aa2d711bdff92dce/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.583356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5966396128491d814472e980cfd873, user_id=43b1ece3cb2e4fd6aa2d711bdff92dce, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.586642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.696161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1227/2430] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:34:40 2022] PUT /identity/v3/projects/cd5966396128491d814472e980cfd873/users/43b1ece3cb2e4fd6aa2d711bdff92dce/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:34:40.701501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c02d9-2ec2-4a6a-a245-01eeb7d8278a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.701988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c02d9-2ec2-4a6a-a245-01eeb7d8278a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.702277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c02d9-2ec2-4a6a-a245-01eeb7d8278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.720772 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-bf0c02d9-2ec2-4a6a-a245-01eeb7d8278a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:40.722110 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bf0c02d9-2ec2-4a6a-a245-01eeb7d8278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:40.781675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1204/2431] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:40.791368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:40.793033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:40.800043 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:40.858875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0be53-39c3-4a74-8eff-e0aa43557f6d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:40.864272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:40.864956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:40.865239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:40.865517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:40.876880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:40.878581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:40.888504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1228/2432] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:34:46.524030 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:46.524534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:46.524705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:46.556839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] MFA Rules not processed for user `43b1ece3cb2e4fd6aa2d711bdff92dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:46.558117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:46.630941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1205/2433] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:34:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:46.635768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:46.646381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e84a78-4414-4817-b44f-93632906386d service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:46.651086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:46.651825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:46.652118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:46.652397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:46.653094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:46.655033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:46.754120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1229/2434] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:34:46 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:34:46.758490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:46.774556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da074fa8-e4f8-4c89-bc1b-822852b4c423 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:46.835455 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:46.836603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:46.837111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:46.837577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:46.838877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:46.841580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:46.927259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:46.934295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1206/2435] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:34:46 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:34:46.938540 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:46.948912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842a47-2f33-4d96-8c13-5476a6f00450 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:46.954316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:46.955055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:46.955369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:46.955646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:46.956372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:46.958035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:46.969496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1230/2436] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:34:46.973681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:46.984032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f2b62-fd80-47d1-b8be-581a70cfcb87 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:46.989338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:46.990231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:46.990550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:46.990832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] PATH_INFO: `/v3/projects/98d44b4a07444128a45ad8eaed75bfa4/users/c83e7f77ec3c41ff8478f0b955a4c3e7/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:47.006075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d44b4a07444128a45ad8eaed75bfa4, user_id=c83e7f77ec3c41ff8478f0b955a4c3e7, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:47.008528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:47.120829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1207/2437] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:34:46 2022] PUT /identity/v3/projects/98d44b4a07444128a45ad8eaed75bfa4/users/c83e7f77ec3c41ff8478f0b955a4c3e7/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:34:47.125644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:47.136646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd3276-2c20-46cf-9da8-81e442518136 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:47.195542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:47.196155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:47.196475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:47.196715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:47.197436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:47.199038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:47.209492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1231/2438] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:34:47.214951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:47.228797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8cca3-ae11-4291-8794-1ff3057d958a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:47.235265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509bd95-3467-4859-a36a-9a3c84399821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:47.236774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509bd95-3467-4859-a36a-9a3c84399821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:47.237221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509bd95-3467-4859-a36a-9a3c84399821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:47.237669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509bd95-3467-4859-a36a-9a3c84399821 admin admin] PATH_INFO: `/v3/projects/98d44b4a07444128a45ad8eaed75bfa4/users/c83e7f77ec3c41ff8478f0b955a4c3e7/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:47.240156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509bd95-3467-4859-a36a-9a3c84399821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d44b4a07444128a45ad8eaed75bfa4, user_id=c83e7f77ec3c41ff8478f0b955a4c3e7, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:47.242808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509bd95-3467-4859-a36a-9a3c84399821 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:47.253369 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1208/2439] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:34:47 2022] PUT /identity/v3/projects/98d44b4a07444128a45ad8eaed75bfa4/users/c83e7f77ec3c41ff8478f0b955a4c3e7/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:34:48.073368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:48.075657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:48.162997 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6637-9267-40bb-a498-0fca379a8f8a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:48.171023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:48.252963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:48.253698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:48.253982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:48.254254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:48.256077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:48.315611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:48.317215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:48.327101 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1232/2440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:34:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:34:53.346278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764d5d4-fdc3-40aa-978f-5603acb241f7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:53.346860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764d5d4-fdc3-40aa-978f-5603acb241f7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:53.347103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764d5d4-fdc3-40aa-978f-5603acb241f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:53.390655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1764d5d4-fdc3-40aa-978f-5603acb241f7 None None] MFA Rules not processed for user `c83e7f77ec3c41ff8478f0b955a4c3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:53.393049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1764d5d4-fdc3-40aa-978f-5603acb241f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:53.514528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1209/2441] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:34:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:53.530177 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:53.533436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:53.632525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9e610-c100-42fc-bdd0-8f87fbddc41c service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:53.652486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:53.653316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:53.653316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:53.653506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:53.749428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:53.752448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:53.758862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:34:53.764034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1233/2442] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:34:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:34:54.033309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:54.033657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:54.034057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:54.054728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:34:54.055958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:54.124180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1210/2443] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:34:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:34:54.336158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:54.337641 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:54.420543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a192e7-4fb5-4d64-b2fb-f475e31b166c service glance] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:54.425991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861502e5-e3ad-4a4f-8eea-78d746cb971a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:54.426649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861502e5-e3ad-4a4f-8eea-78d746cb971a service swift] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:54.426933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861502e5-e3ad-4a4f-8eea-78d746cb971a service swift] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:54.427213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861502e5-e3ad-4a4f-8eea-78d746cb971a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:54.436591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861502e5-e3ad-4a4f-8eea-78d746cb971a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:54.438256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861502e5-e3ad-4a4f-8eea-78d746cb971a service swift] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:54.448360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1234/2444] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:34:57.498995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:34:57.523270 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c1a9e-9275-42ea-9ce4-33e5495be3fa None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:34:57.542880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:34:57.543682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:34:57.543975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:34:57.544260 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:34:57.569760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:34:57.602709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:34:57.604725 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:34:57.626799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1211/2445] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:34:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:03.987801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806bc62-b39f-4057-8588-ed07e8c13263 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:03.987801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806bc62-b39f-4057-8588-ed07e8c13263 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:03.987801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806bc62-b39f-4057-8588-ed07e8c13263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:04.008218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6806bc62-b39f-4057-8588-ed07e8c13263 None None] MFA Rules not processed for user `c052b2f518064843928b4eff569e913f`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:04.013598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6806bc62-b39f-4057-8588-ed07e8c13263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:04.112614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1235/2446] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:04.603542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:04.697680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:04.717133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c072f1-8e4c-4f3c-a63e-1011454ba44a service nova] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:04.728798 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:04.729307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:04.729745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:04.730082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:04.734022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:04.747811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:04.749910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd7c39-5410-45ef-a0cc-de10a2da5445 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:04.760557 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1212/2447] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:07.654852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:07.655470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:07.655470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:07.674875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] MFA Rules not processed for user `43b1ece3cb2e4fd6aa2d711bdff92dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:07.676121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:07.683212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1236/2448] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:11.794247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:11.794790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:11.794897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:11.817335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:11.818652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:11.826105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1213/2449] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:35:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:11.838537 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:11.852732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469a7ab-f2d3-41bc-b71f-7d89b2f3f12a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:11.859557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:11.860604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:11.860981 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:11.861422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:11.873243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:11.873243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:11.886151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1237/2450] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:35:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:35:12.004632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:12.006158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:12.020703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c984152-3c1f-46ca-8bb6-68ee43ca7822 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:12.027185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:12.027185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:12.027185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:12.027185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:12.027564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:12.154108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:12.157730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:12.181790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1214/2451] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:35:20.041846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:20.043440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:20.063076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd94c5d-160f-4659-a744-c04de83ed3c7 service swift] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:20.068079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:20.069169 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:20.069642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:20.069971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] PATH_INFO: `/v3/users/12816eca659542af9e41b92fdb674eab` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:20.071245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12816eca659542af9e41b92fdb674eab)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:20.073235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:20.373951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12816eca659542af9e41b92fdb674eab'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.375042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12816eca659542af9e41b92fdb674eab'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.393170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12816eca659542af9e41b92fdb674eab'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.397688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:20.400709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12816eca659542af9e41b92fdb674eab'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.473553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-acd6403a-14eb-451b-9aae-9c208358bcb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12816eca659542af9e41b92fdb674eab'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.480284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1238/2452] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:35:20 2022] DELETE /identity/v3/users/12816eca659542af9e41b92fdb674eab => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:20.718886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:20.725281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74718e60-ce29-4455-8323-723c0667af99 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:20.788291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:20.789241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:20.789525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:20.789801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] PATH_INFO: `/v3/projects/bda53dca70e54e208b2767a833923234` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:20.790984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda53dca70e54e208b2767a833923234)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:20.792976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:20.909197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda53dca70e54e208b2767a833923234', 'request_id': 'req-1f497013-afde-4dde-b441-d7fd9f03d75f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.974835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda53dca70e54e208b2767a833923234', 'request_id': 'req-1f497013-afde-4dde-b441-d7fd9f03d75f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.981734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda53dca70e54e208b2767a833923234', 'request_id': 'req-1f497013-afde-4dde-b441-d7fd9f03d75f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.990365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:20.995162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] The token cache is being invalidate because project bda53dca70e54e208b2767a833923234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:35:20.995481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.996399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f497013-afde-4dde-b441-d7fd9f03d75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:20.997793 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1215/2453] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:35:20 2022] DELETE /identity/v3/projects/bda53dca70e54e208b2767a833923234 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:21.004207 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.004498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.004831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.023761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:21.024951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:21.090067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1239/2454] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:35:21.096393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.096713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.096997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.116211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:21.118588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:21.125294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1216/2455] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:21.129664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.139424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492747-5831-42b6-8fd3-3b5905f0c09c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.144200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.144858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.145147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.145428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.146157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:35:21.146538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.148401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.160907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1240/2456] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:35:21.165169 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.176235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cbd27-b482-4212-9128-f6d16de7c71e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.180855 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.181501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.181784 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.182063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.182758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.184722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.240730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1217/2457] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:35:21 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:35:21.247651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.258472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff15e4-a6b7-4d9c-9ccc-8b239a65df89 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.317120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.317808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.318093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.318372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.319122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.321084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.388308 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1241/2458] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:35:21 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:35:21.393680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.404542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a5652-cbb1-4dd1-b1e4-096ad0f6ec76 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.409156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.409840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.410122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.410419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.411172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.412756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.423129 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1218/2459] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:35:21.427131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.437690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0dca8a-1336-4eac-9110-8530e3673e36 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.442292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc2ccf-c784-4bf5-9e75-41612297a957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.442941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc2ccf-c784-4bf5-9e75-41612297a957 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.443227 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc2ccf-c784-4bf5-9e75-41612297a957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.443552 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc2ccf-c784-4bf5-9e75-41612297a957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.444271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc2ccf-c784-4bf5-9e75-41612297a957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.445868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc2ccf-c784-4bf5-9e75-41612297a957 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.455763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1242/2460] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:35:21.459750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.469463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7417c-9c1c-4579-8819-77d5f89cf0cd admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.474024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.474894 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.475179 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.475512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] PATH_INFO: `/v3/projects/8333093d796544d3962a75a59404a613/users/db7952d498e64c9db76878af469ccaf6/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.490874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8333093d796544d3962a75a59404a613, user_id=db7952d498e64c9db76878af469ccaf6, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.493554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.535825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1219/2461] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:35:21 2022] PUT /identity/v3/projects/8333093d796544d3962a75a59404a613/users/db7952d498e64c9db76878af469ccaf6/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:21.546376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.546765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.547074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.573543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] MFA Rules not processed for user `db7952d498e64c9db76878af469ccaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:21.574758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:21.616123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:21.642084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1243/2462] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:21.653669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:21.656009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:21.734153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de1694-c49a-4505-8cfa-d5c94942fd66 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:21.740403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266aea68-3fd2-40e5-a63c-e07e92c31986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:21.741065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266aea68-3fd2-40e5-a63c-e07e92c31986 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:21.741352 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266aea68-3fd2-40e5-a63c-e07e92c31986 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:21.741631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266aea68-3fd2-40e5-a63c-e07e92c31986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:21.750379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266aea68-3fd2-40e5-a63c-e07e92c31986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:21.752016 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266aea68-3fd2-40e5-a63c-e07e92c31986 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:21.761336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1220/2463] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:26.041100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:26.101794 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d831871-2fee-4fca-9c7a-de6c2d1fd77b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:26.106375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:26.107257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:26.108448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:26.108448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] PATH_INFO: `/v3/users/db7952d498e64c9db76878af469ccaf6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:26.109068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7952d498e64c9db76878af469ccaf6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:26.111264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:26.247943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7952d498e64c9db76878af469ccaf6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:26.248575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7952d498e64c9db76878af469ccaf6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:26.260881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7952d498e64c9db76878af469ccaf6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:26.267746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7952d498e64c9db76878af469ccaf6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:26.357402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7952d498e64c9db76878af469ccaf6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:26.365050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1244/2464] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:35:26 2022] DELETE /identity/v3/users/db7952d498e64c9db76878af469ccaf6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:26.369599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:26.370102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:26.372595 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:26.387699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:26.388959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:26.450856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1221/2465] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:26.461201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:26.462612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:26.520440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d096d3a-3854-44f0-9647-6abd33c7cf2c admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:26.523926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ba1a5-9c47-41b2-9e85-2eee34882239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:26.524571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba1a5-9c47-41b2-9e85-2eee34882239 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:26.524857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba1a5-9c47-41b2-9e85-2eee34882239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:26.525131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ba1a5-9c47-41b2-9e85-2eee34882239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:26.534512 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba1a5-9c47-41b2-9e85-2eee34882239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:26.535964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ba1a5-9c47-41b2-9e85-2eee34882239 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:26.545082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1245/2466] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:26.954061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:26.956036 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:26.968367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d92669-432f-4afa-b20d-97eaa64cbd42 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:26.974443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:26.975241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:26.975561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:26.975843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] PATH_INFO: `/v3/projects/8333093d796544d3962a75a59404a613` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:26.977042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8333093d796544d3962a75a59404a613)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:26.978948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.001674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:27.119170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8333093d796544d3962a75a59404a613', 'request_id': 'req-343aaf43-24e8-4bb8-8417-38db5a570df0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:27.187264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8333093d796544d3962a75a59404a613', 'request_id': 'req-343aaf43-24e8-4bb8-8417-38db5a570df0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:27.193189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8333093d796544d3962a75a59404a613', 'request_id': 'req-343aaf43-24e8-4bb8-8417-38db5a570df0'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:27.204809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] The token cache is being invalidate because project 8333093d796544d3962a75a59404a613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:35:27.205103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:27.205781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-343aaf43-24e8-4bb8-8417-38db5a570df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:35:27.207561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1222/2467] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:35:26 2022] DELETE /identity/v3/projects/8333093d796544d3962a75a59404a613 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:27.212420 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.218487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.218487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.220800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:27.231825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:27.233026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:27.294353 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1246/2468] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:35:27.302147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.302464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.302747 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.321266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:27.322480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:27.330981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1223/2469] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:35:27.335348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:27.345650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f60a-dcf3-4ac3-9262-22b025d836e9 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:27.350401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:27.351103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.351469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.351623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.352343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:35:27.352647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:27.354277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.366404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1247/2470] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:35:27.371229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:27.380731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d14e8-4e74-4d7f-a07b-6787b7c97cf0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:27.386938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:27.387646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.387962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.388247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.388988 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:27.391484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.493223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1224/2471] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:35:27 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:35:27.497103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:27.507294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ae9f-2943-4e7a-bfb9-50c3f519b440 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:27.567170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:27.567888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.568175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.568468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.569156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:27.571151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.674000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1248/2472] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:35:27 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:35:27.678233 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:27.689340 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde807-87ac-43cb-8241-78d8e04dd536 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:27.693962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:27.694619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.694902 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.695175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.696123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:27.697841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.711111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1225/2473] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:35:27.715392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:27.725833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77e8b7-7af6-4ff8-87b0-8cc8222f713e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:27.730663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e085161-e0d6-46fc-9ff8-fe5b7120f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:27.731356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e085161-e0d6-46fc-9ff8-fe5b7120f0ef admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.731648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e085161-e0d6-46fc-9ff8-fe5b7120f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.731928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e085161-e0d6-46fc-9ff8-fe5b7120f0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.732651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e085161-e0d6-46fc-9ff8-fe5b7120f0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:27.734403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e085161-e0d6-46fc-9ff8-fe5b7120f0ef admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.744822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1249/2474] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:35:27.749706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:27.759617 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da50f91-ff67-48a2-98ae-434ccbcf735f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:27.764136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:27.765642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.765924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.766199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] PATH_INFO: `/v3/projects/713d8f0f528a413f86bb0dd720efd2ad/users/a1bbc63fdd7c46a49b78253acadbff7e/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.780966 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713d8f0f528a413f86bb0dd720efd2ad, user_id=a1bbc63fdd7c46a49b78253acadbff7e, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:27.783629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:27.854847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1226/2475] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:35:27 2022] PUT /identity/v3/projects/713d8f0f528a413f86bb0dd720efd2ad/users/a1bbc63fdd7c46a49b78253acadbff7e/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:35:27.868067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:27.868390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:27.868677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:27.904056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] MFA Rules not processed for user `a1bbc63fdd7c46a49b78253acadbff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:27.904056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:27.997037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1250/2476] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:27.998108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:28.001963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.074890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcdfd0-bc0f-4b44-bfaa-a6c25d85ddc4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.080463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.081231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.082005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.082005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.082474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.084432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.172382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1227/2477] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:35:28 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:35:28.176265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.187142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49bc1c-2afb-4d4c-a327-d4f1ec76f6a9 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.242737 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.243442 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.243715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.243995 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.244693 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.246657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.318832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1251/2478] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:35:28 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:35:28.322968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.333760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce6d0d-5685-4356-b217-2561088a6bde admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.338300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.338979 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.339259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.339570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.340315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.341904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.353128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1228/2479] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:35:28.357368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.368016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fbb28-2477-4b27-927a-7f8956ec15a7 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.372520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.373394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.373679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.374464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] PATH_INFO: `/v3/projects/3085a7b9cbd7492b828327f737d80084/users/82957a1878014920ab1086a238a3e4d7/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.390917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3085a7b9cbd7492b828327f737d80084, user_id=82957a1878014920ab1086a238a3e4d7, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.393764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.491218 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1252/2480] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:35:28 2022] PUT /identity/v3/projects/3085a7b9cbd7492b828327f737d80084/users/82957a1878014920ab1086a238a3e4d7/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:28.495097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.505566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9c5b4-6e7f-4b94-9fe4-c0bc7ffd2ba8 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.514189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:28.567954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b63d-491a-45be-b1ed-15d0d64eb4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.568633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b63d-491a-45be-b1ed-15d0d64eb4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.568920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b63d-491a-45be-b1ed-15d0d64eb4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.569199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b63d-491a-45be-b1ed-15d0d64eb4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.569929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b63d-491a-45be-b1ed-15d0d64eb4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.571570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b63d-491a-45be-b1ed-15d0d64eb4e6 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.582532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1229/2481] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:35:28.586302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.596451 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65adb3a-32b9-4a54-91ef-1faf30292701 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.601050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.601918 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.602201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.602474 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] PATH_INFO: `/v3/projects/3085a7b9cbd7492b828327f737d80084/users/82957a1878014920ab1086a238a3e4d7/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.604434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3085a7b9cbd7492b828327f737d80084, user_id=82957a1878014920ab1086a238a3e4d7, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.607039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.618816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1253/2482] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:35:28 2022] PUT /identity/v3/projects/3085a7b9cbd7492b828327f737d80084/users/82957a1878014920ab1086a238a3e4d7/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:35:28.628215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.628511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.628795 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.654991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] MFA Rules not processed for user `82957a1878014920ab1086a238a3e4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:35:28.656532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:28.737696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1230/2483] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:35:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:35:28.750301 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:28.751663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:28.810496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b447584-2ab3-41b0-b75f-af6292f866b9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:28.814811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:28.815498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:28.815821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:28.816105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:28.827504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:28.831581 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:28.838591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1254/2484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:29.053178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:29.063860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5844cac-b59e-476c-ac3f-86f332fe387a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:29.067549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:29.068225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:29.068511 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:29.068819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:29.134681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:29.136188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:29.148541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1231/2485] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:38.834248 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:39.142704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=85 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:35:48.911815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:35:48.919437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:48.970399 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ada2b8-73bd-47d0-a252-fbbb228bcc8e service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:35:48.974137 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:35:48.974782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:35:48.975061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:35:48.975367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:35:48.977116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:35:49.041319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:35:49.042815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:35:49.052506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1255/2486] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:35:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:35:59.048127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=50 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:26.380282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:26.462656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed3ae6-47c7-44b0-88f2-bc0644175dc6 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:26.467647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:26.468553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:26.468843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:26.469123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] PATH_INFO: `/v3/users/a1bbc63fdd7c46a49b78253acadbff7e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:26.470358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bbc63fdd7c46a49b78253acadbff7e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:26.472470 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:26.496715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:26.498532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:26.585940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a536428-0aa0-4b75-9a0b-a677190e0668 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:26.593412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:26.594598 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:26.597725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:26.597725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:26.597725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:26.609808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:26.611424 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7455d3-97c8-489f-accf-51498ebcf974 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:26.622360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1256/2487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:36:26.659187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bbc63fdd7c46a49b78253acadbff7e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:26.689143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bbc63fdd7c46a49b78253acadbff7e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:26.690580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bbc63fdd7c46a49b78253acadbff7e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:26.712006 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bbc63fdd7c46a49b78253acadbff7e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:26.779928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bbc63fdd7c46a49b78253acadbff7e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:26.786850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1232/2488] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:26 2022] DELETE /identity/v3/users/a1bbc63fdd7c46a49b78253acadbff7e => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:36:26.792339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:26.792688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:26.793070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:26.816021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:26.817242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:26.891893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1257/2489] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:36:26.901765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:26.903315 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:26.964096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3984ff-05f8-46f2-bde6-e0e346d429e2 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:26.970643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:26.971354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:26.971643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:26.971918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:26.981682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:26.983126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:26.993969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1233/2490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:27.485731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:27.487180 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:27.499359 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be77aa2-d9f3-4579-8ac6-2f9df034dd15 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:27.504730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:27.505605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:27.505891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:27.506174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] PATH_INFO: `/v3/projects/713d8f0f528a413f86bb0dd720efd2ad` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:27.507400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=713d8f0f528a413f86bb0dd720efd2ad)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:27.511633 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:27.645197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713d8f0f528a413f86bb0dd720efd2ad', 'request_id': 'req-942ef347-44fa-4379-a040-22b1f6f28c8d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.650832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713d8f0f528a413f86bb0dd720efd2ad', 'request_id': 'req-942ef347-44fa-4379-a040-22b1f6f28c8d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.729208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713d8f0f528a413f86bb0dd720efd2ad', 'request_id': 'req-942ef347-44fa-4379-a040-22b1f6f28c8d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.741009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] The token cache is being invalidate because project 713d8f0f528a413f86bb0dd720efd2ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:27.741300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.741930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.743535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1258/2491] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:27 2022] DELETE /identity/v3/projects/713d8f0f528a413f86bb0dd720efd2ad => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:27.747564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:27.757811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae538ca3-9765-440a-889b-9e8ef4155c17 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:27.776155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:27.814439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:27.815383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:27.815671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:27.816220 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] PATH_INFO: `/v3/users/82957a1878014920ab1086a238a3e4d7` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:27.817324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82957a1878014920ab1086a238a3e4d7)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:27.820293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:27.987553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82957a1878014920ab1086a238a3e4d7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.994727 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82957a1878014920ab1086a238a3e4d7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:27.995460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82957a1878014920ab1086a238a3e4d7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.008656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82957a1878014920ab1086a238a3e4d7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.082107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-b0656fc7-4982-432c-8ebc-fca32d8966d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82957a1878014920ab1086a238a3e4d7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.088542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1234/2492] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:27 2022] DELETE /identity/v3/users/82957a1878014920ab1086a238a3e4d7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:36:28.510733 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:28.512225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:28.527453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef347-44fa-4379-a040-22b1f6f28c8d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:28.545876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:28.604193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:28.605650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:28.606094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:28.606602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] PATH_INFO: `/v3/projects/3085a7b9cbd7492b828327f737d80084` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:28.608648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3085a7b9cbd7492b828327f737d80084)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:28.612519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:28.763483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3085a7b9cbd7492b828327f737d80084', 'request_id': 'req-87f832cb-60f9-4947-b258-e6b53e3342ad'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.768971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3085a7b9cbd7492b828327f737d80084', 'request_id': 'req-87f832cb-60f9-4947-b258-e6b53e3342ad'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.832450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3085a7b9cbd7492b828327f737d80084', 'request_id': 'req-87f832cb-60f9-4947-b258-e6b53e3342ad'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.850167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] The token cache is being invalidate because project 3085a7b9cbd7492b828327f737d80084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:28.850678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.851577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-87f832cb-60f9-4947-b258-e6b53e3342ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:28.853555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1259/2493] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:28 2022] DELETE /identity/v3/projects/3085a7b9cbd7492b828327f737d80084 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:28.860945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:28.864650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:28.864650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.021384 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:29.022526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:29.106958 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1235/2494] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:29.116836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.117121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.117587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.138075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:29.139288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:29.147893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1260/2495] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:29.152248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.166172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768fb3f-5edb-4aa6-b15d-5720323661c8 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.168338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.169067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.170009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.170009 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.170780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:36:29.170780 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.172208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:29.184890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1236/2496] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:36:29.192262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.202106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deccf00-d38a-472f-b859-b81525bf371c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.207979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.208648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.208935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.209214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.209925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.211944 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:29.325056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1261/2497] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:29 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:36:29.330252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.340432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45166f3-3183-4ef7-99b0-fb8a6468e6f3 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.369040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:29.403208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.403926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.404217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.404496 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.405198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.407176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:29.493545 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1237/2498] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:29 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:36:29.497784 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.508888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c577bd6-9a6b-402b-81fd-c00479112c15 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.513563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.514229 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.514516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.514798 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.515770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.517375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:29.531380 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1262/2499] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:29.532517 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.542298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c235f4-3369-4b94-b4f0-1df62359a857 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.547426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa34de-6e4f-40af-8f52-4c2a4ff2058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.548079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa34de-6e4f-40af-8f52-4c2a4ff2058a admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.548365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa34de-6e4f-40af-8f52-4c2a4ff2058a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.548641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa34de-6e4f-40af-8f52-4c2a4ff2058a admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.552235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa34de-6e4f-40af-8f52-4c2a4ff2058a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.552235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa34de-6e4f-40af-8f52-4c2a4ff2058a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:29.560689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1238/2500] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:29.565590 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.576755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779fee3-eda5-4d92-984c-b4574dc317a9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.581391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.582271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.582557 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.582839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] PATH_INFO: `/v3/projects/eafdfb4387414dbd84d23f403ea277ac/users/ad8325afb02c4fa2b1fb86e461476970/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.600900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eafdfb4387414dbd84d23f403ea277ac, user_id=ad8325afb02c4fa2b1fb86e461476970, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.601445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:29.688777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1263/2501] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:29 2022] PUT /identity/v3/projects/eafdfb4387414dbd84d23f403ea277ac/users/ad8325afb02c4fa2b1fb86e461476970/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:29.699102 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.699447 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.699736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.729245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] MFA Rules not processed for user `ad8325afb02c4fa2b1fb86e461476970`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:29.730473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:29.812807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1239/2502] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:29.817869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:29.866279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:29.886373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ff5ca-7a56-4bff-8fe0-d225cef8c0cb admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:29.891742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:29.892391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:29.894262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:29.894262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:29.894262 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:29.895531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.000367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1264/2503] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:29 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:30.004542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.016218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f5847-367d-47ce-bb0b-9857b40e313a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.072432 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.073122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.073409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.073685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.074382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.076336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.282261 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1240/2504] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:30 2022] POST /identity/v3/users => generated 300 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:30.294105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.297332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9348543-c77f-430a-86dd-194dd410df81 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.301994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.302651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.302933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.303255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.304005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.305603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.319198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1265/2505] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:30.322844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.333640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71759ebe-598b-4a64-8b01-097bf4fd5037 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.339040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.340172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.340295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.340539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] PATH_INFO: `/v3/projects/ec804fb2ffb64a34aa37023fd7d4d559/users/6f3847266cf742cb9e223a8fc95bc650/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.357640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec804fb2ffb64a34aa37023fd7d4d559, user_id=6f3847266cf742cb9e223a8fc95bc650, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.360410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.437498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1241/2506] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:30 2022] PUT /identity/v3/projects/ec804fb2ffb64a34aa37023fd7d4d559/users/6f3847266cf742cb9e223a8fc95bc650/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:30.441432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.451480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaa5df-f9fb-4eb8-8d4e-f0ece152a79d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.509792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88871e-d7ff-4b69-aa3c-0cb98256ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.510476 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88871e-d7ff-4b69-aa3c-0cb98256ba3d admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.510744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88871e-d7ff-4b69-aa3c-0cb98256ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.511054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88871e-d7ff-4b69-aa3c-0cb98256ba3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.511814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88871e-d7ff-4b69-aa3c-0cb98256ba3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.513419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88871e-d7ff-4b69-aa3c-0cb98256ba3d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.524042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1266/2507] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:30.528162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.534521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:30.539959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388aa06-fa72-480c-8ef6-f620c386615c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.544817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.545700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.545984 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.546273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] PATH_INFO: `/v3/projects/ec804fb2ffb64a34aa37023fd7d4d559/users/6f3847266cf742cb9e223a8fc95bc650/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.548240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec804fb2ffb64a34aa37023fd7d4d559, user_id=6f3847266cf742cb9e223a8fc95bc650, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.550987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.562659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1242/2508] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:30 2022] PUT /identity/v3/projects/ec804fb2ffb64a34aa37023fd7d4d559/users/6f3847266cf742cb9e223a8fc95bc650/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:30.573913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.574209 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.574893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.602947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] MFA Rules not processed for user `6f3847266cf742cb9e223a8fc95bc650`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:30.604190 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:30.677559 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1267/2509] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:30.689278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.690971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:30.748980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411be168-fff5-4335-96f9-4c8637c88aff admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.753244 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d82a53-5998-4809-a6f9-fcfb12690650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.753898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82a53-5998-4809-a6f9-fcfb12690650 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.754187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82a53-5998-4809-a6f9-fcfb12690650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.754469 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d82a53-5998-4809-a6f9-fcfb12690650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.763681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d82a53-5998-4809-a6f9-fcfb12690650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.765781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d82a53-5998-4809-a6f9-fcfb12690650 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:30.775477 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1243/2510] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:30.842486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:30.883931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:30.907841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1270bb3-d910-42b4-a587-e0a89ab08aa0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:30.913112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:30.914091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:30.914379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:30.914685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] PATH_INFO: `/v3/users/ad8325afb02c4fa2b1fb86e461476970` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:30.915937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8325afb02c4fa2b1fb86e461476970)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:30.917885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:31.223486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8325afb02c4fa2b1fb86e461476970'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:31.224082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8325afb02c4fa2b1fb86e461476970'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:31.236676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8325afb02c4fa2b1fb86e461476970'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:31.243428 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8325afb02c4fa2b1fb86e461476970'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:31.306496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8325afb02c4fa2b1fb86e461476970'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:31.314083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1268/2511] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:30 2022] DELETE /identity/v3/users/ad8325afb02c4fa2b1fb86e461476970 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:31.318711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca4a27-b1f6-4c6c-8d26-26774520ee2d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:31.319237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca4a27-b1f6-4c6c-8d26-26774520ee2d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:31.319378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca4a27-b1f6-4c6c-8d26-26774520ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:31.337597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-78ca4a27-b1f6-4c6c-8d26-26774520ee2d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:31.340436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-78ca4a27-b1f6-4c6c-8d26-26774520ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:31.398788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1244/2512] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:31.409711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:31.411372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:31.477440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a59dbc-df9a-43be-a147-36e05ab921ec admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:31.481175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:31.481847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:31.485647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:31.485647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:31.493782 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:31.495446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:31.506935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1269/2513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:36:31.578422 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:31.578735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:31.578983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:31.597553 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] MFA Rules not processed for user `a0a147daff9846e699c26259fac4e004`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:31.598797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:31.663194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1245/2514] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:36:32.073292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:32.074767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:32.088103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0838181-01ee-436f-beb8-d28395da668f service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:32.092755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:32.094898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:32.095182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:32.095516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] PATH_INFO: `/v3/projects/eafdfb4387414dbd84d23f403ea277ac` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:32.096724 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eafdfb4387414dbd84d23f403ea277ac)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:32.098652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:32.251197 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eafdfb4387414dbd84d23f403ea277ac', 'request_id': 'req-7a60e059-1478-4311-bd7c-e46ee81433e6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.253710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:32.257575 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eafdfb4387414dbd84d23f403ea277ac', 'request_id': 'req-7a60e059-1478-4311-bd7c-e46ee81433e6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.333713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eafdfb4387414dbd84d23f403ea277ac', 'request_id': 'req-7a60e059-1478-4311-bd7c-e46ee81433e6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.346419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] The token cache is being invalidate because project eafdfb4387414dbd84d23f403ea277ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:32.346712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.347371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.348901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1270/2515] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:32 2022] DELETE /identity/v3/projects/eafdfb4387414dbd84d23f403ea277ac => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:32.352882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:32.364153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da68b1-105e-4b6c-a811-afd0a5ad670f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:32.397699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:32.420600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:32.422079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:32.422357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:32.422857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] PATH_INFO: `/v3/users/6f3847266cf742cb9e223a8fc95bc650` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:32.423823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3847266cf742cb9e223a8fc95bc650)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:32.425734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:32.611552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3847266cf742cb9e223a8fc95bc650'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.618788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3847266cf742cb9e223a8fc95bc650'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.619968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3847266cf742cb9e223a8fc95bc650'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.631977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3847266cf742cb9e223a8fc95bc650'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.729357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-a6fe173f-c72a-4a29-8fc3-59ac46661b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3847266cf742cb9e223a8fc95bc650'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:32.736120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1246/2516] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:32 2022] DELETE /identity/v3/users/6f3847266cf742cb9e223a8fc95bc650 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:33.193821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:33.195347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:33.209230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60e059-1478-4311-bd7c-e46ee81433e6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:33.269011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:33.269879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:33.270173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:33.270467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] PATH_INFO: `/v3/projects/ec804fb2ffb64a34aa37023fd7d4d559` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:33.271691 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec804fb2ffb64a34aa37023fd7d4d559)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:33.279647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:33.403467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec804fb2ffb64a34aa37023fd7d4d559', 'request_id': 'req-c604884a-3acc-4a7b-96c2-2e7cdd94da10'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:33.409992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec804fb2ffb64a34aa37023fd7d4d559', 'request_id': 'req-c604884a-3acc-4a7b-96c2-2e7cdd94da10'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:33.493922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec804fb2ffb64a34aa37023fd7d4d559', 'request_id': 'req-c604884a-3acc-4a7b-96c2-2e7cdd94da10'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:33.506186 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] The token cache is being invalidate because project ec804fb2ffb64a34aa37023fd7d4d559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:33.506480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:33.507121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-c604884a-3acc-4a7b-96c2-2e7cdd94da10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:33.508704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1271/2517] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:33 2022] DELETE /identity/v3/projects/ec804fb2ffb64a34aa37023fd7d4d559 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:33.514479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:33.515499 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:33.515776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:33.536290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:33.537669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:33.608146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1247/2518] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:33.616300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:33.616601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:33.616892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:33.636514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:33.637719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:33.646328 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1272/2519] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:33.650863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:33.660857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7bb0c-a433-497e-9d56-f3fd63c37df0 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:33.665451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:33.666113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:33.666396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:33.666671 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:33.667420 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:36:33.667726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:33.669578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:33.681846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1248/2520] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:36:33.686331 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:33.700278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08ae36-5352-4377-9b4d-fa11c5c509cc None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:33.705726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:33.706376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:33.706660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:33.706938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:33.707672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:33.709589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:33.817097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1273/2521] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:33 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:33.821316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:33.831786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1547b6-afe0-4b38-8034-6da4b6797ac5 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:33.878546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:33.894640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:33.895353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:33.895645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:33.895925 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:33.896658 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:33.898602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:33.980832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1249/2522] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:33 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:33.985716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:33.996061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ee4cc-b4c7-494e-9327-75bec7c95e67 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.001318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.001987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.002261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.002538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.003265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.004925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.019443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1274/2523] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:34.020572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.031631 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d8ebd-c19e-435a-b793-1c7a7fcf4e77 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.039921 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de0eaa-63e2-4644-b295-92ff3ec1ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.045044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de0eaa-63e2-4644-b295-92ff3ec1ff21 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.045044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de0eaa-63e2-4644-b295-92ff3ec1ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.045044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de0eaa-63e2-4644-b295-92ff3ec1ff21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.045044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de0eaa-63e2-4644-b295-92ff3ec1ff21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.045044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de0eaa-63e2-4644-b295-92ff3ec1ff21 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.056015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1250/2524] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:34.059384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.073676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc7015-a20d-4714-b616-9fa6bc17a354 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.076333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.077251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.078116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.078403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] PATH_INFO: `/v3/projects/e435dbe7f09c498eb6ed139ca7c92710/users/5d412e199692481191686cf83418f69d/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.094462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e435dbe7f09c498eb6ed139ca7c92710, user_id=5d412e199692481191686cf83418f69d, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.097279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.165103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1275/2525] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:34 2022] PUT /identity/v3/projects/e435dbe7f09c498eb6ed139ca7c92710/users/5d412e199692481191686cf83418f69d/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:34.175823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.176119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.176454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.203050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] MFA Rules not processed for user `5d412e199692481191686cf83418f69d`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:34.204343 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:34.269108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1251/2526] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:34.273626 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.283191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:34.335992 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d7517-43de-45d6-8f38-f139bc16dc19 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.340630 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.341285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.341569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.341846 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.342530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.345075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.434091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1276/2527] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:34 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:34.438713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.449900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d7d6f-523b-4fd4-88d1-78ff4644df96 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.511411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.512106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.512391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.512670 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.513363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.516065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.604013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1252/2528] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:34 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:34.607747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.619572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa6a3-a7e4-46be-a7cd-7a11c359c7ad admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.625339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.629940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.629940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.629940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.629940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.629940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.642321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1277/2529] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:34.646440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.664341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abed653-bc60-489b-a311-f35302cc5e93 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.671284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.673212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.673212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.673212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] PATH_INFO: `/v3/projects/1dedf3249e334c8bb12b1fa29af7c07f/users/0bd12b1d2f2649b181184be4d3db800b/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.690678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dedf3249e334c8bb12b1fa29af7c07f, user_id=0bd12b1d2f2649b181184be4d3db800b, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.693378 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.764518 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1253/2530] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:34 2022] PUT /identity/v3/projects/1dedf3249e334c8bb12b1fa29af7c07f/users/0bd12b1d2f2649b181184be4d3db800b/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:34.781388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.791709 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c085b1-546a-4ba2-a12a-b5c558897fea admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.848927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80b875-4ecf-4f5d-9314-b1fb62dfcaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.849596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80b875-4ecf-4f5d-9314-b1fb62dfcaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.849882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80b875-4ecf-4f5d-9314-b1fb62dfcaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.850159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80b875-4ecf-4f5d-9314-b1fb62dfcaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.850874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80b875-4ecf-4f5d-9314-b1fb62dfcaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.852530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80b875-4ecf-4f5d-9314-b1fb62dfcaaa admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.864861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1278/2531] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:34.869051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:34.880216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ca3cc-8d59-466b-9f4a-965dc056b709 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:34.885170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:34.886172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.886460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.886738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] PATH_INFO: `/v3/projects/1dedf3249e334c8bb12b1fa29af7c07f/users/0bd12b1d2f2649b181184be4d3db800b/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.892824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dedf3249e334c8bb12b1fa29af7c07f, user_id=0bd12b1d2f2649b181184be4d3db800b, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:34.895502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:34.907013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1254/2532] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:34 2022] PUT /identity/v3/projects/1dedf3249e334c8bb12b1fa29af7c07f/users/0bd12b1d2f2649b181184be4d3db800b/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:34.918335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:34.918711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:34.919049 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:34.946763 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] MFA Rules not processed for user `0bd12b1d2f2649b181184be4d3db800b`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:34.948155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:35.027877 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1279/2533] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:35.040307 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:35.042731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:35.051020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:35.109626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae5bbb-adc3-407e-a961-7b0efe051855 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:35.113439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637ee46-a5af-406c-9ab8-7d1f80202d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:35.114029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ee46-a5af-406c-9ab8-7d1f80202d1a service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:35.114326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ee46-a5af-406c-9ab8-7d1f80202d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:35.114586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ee46-a5af-406c-9ab8-7d1f80202d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:35.124932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637ee46-a5af-406c-9ab8-7d1f80202d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:35.126364 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637ee46-a5af-406c-9ab8-7d1f80202d1a service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:35.136748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1255/2534] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:35.283201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:35.288011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:35.354517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a533c-cde1-4421-91e6-c05ff48db259 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:35.363379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:35.369897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:35.369897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:35.369897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] PATH_INFO: `/v3/users/5d412e199692481191686cf83418f69d` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:35.369897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d412e199692481191686cf83418f69d)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:35.369897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:35.587765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d412e199692481191686cf83418f69d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:35.588561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d412e199692481191686cf83418f69d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:35.604755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d412e199692481191686cf83418f69d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:35.614656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d412e199692481191686cf83418f69d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:35.700863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d412e199692481191686cf83418f69d'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:35.708595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1280/2535] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:35 2022] DELETE /identity/v3/users/5d412e199692481191686cf83418f69d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:35.714156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:35.714644 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:35.715111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:35.735678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:35.736899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:35.814503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1256/2536] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:35.828553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:35.829956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:35.901968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a538-b6e5-4fc6-b3b2-afe02994abf4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:35.906134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:35.906874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:35.907105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:35.907487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:35.923878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:35.924522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:35.936269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1281/2537] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:36.438028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:36.440031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:36.455775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce3b51-f453-41ca-a572-0a410327561f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:36.461803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:36.462739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:36.463082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:36.463395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] PATH_INFO: `/v3/projects/e435dbe7f09c498eb6ed139ca7c92710` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:36.464715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e435dbe7f09c498eb6ed139ca7c92710)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:36.467067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:36.570256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e435dbe7f09c498eb6ed139ca7c92710', 'request_id': 'req-5cd8a54b-671c-4251-85a4-4d1bb296e669'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.632076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e435dbe7f09c498eb6ed139ca7c92710', 'request_id': 'req-5cd8a54b-671c-4251-85a4-4d1bb296e669'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.639748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e435dbe7f09c498eb6ed139ca7c92710', 'request_id': 'req-5cd8a54b-671c-4251-85a4-4d1bb296e669'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.654865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] The token cache is being invalidate because project e435dbe7f09c498eb6ed139ca7c92710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:36.655282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.656096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.660785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1257/2538] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:36 2022] DELETE /identity/v3/projects/e435dbe7f09c498eb6ed139ca7c92710 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:36.662600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:36.676375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f335-fdd5-493f-a119-2d72797493cb service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:36.697922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:36.742519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:36.743757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:36.744192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:36.744829 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] PATH_INFO: `/v3/users/0bd12b1d2f2649b181184be4d3db800b` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:36.746245 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd12b1d2f2649b181184be4d3db800b)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:36.748538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:36.951068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd12b1d2f2649b181184be4d3db800b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.951793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd12b1d2f2649b181184be4d3db800b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.964821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd12b1d2f2649b181184be4d3db800b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:36.975516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd12b1d2f2649b181184be4d3db800b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.037708 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-beea7ad4-72b2-4c87-a6be-475a38bfb82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd12b1d2f2649b181184be4d3db800b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.046813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1282/2539] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:36 2022] DELETE /identity/v3/users/0bd12b1d2f2649b181184be4d3db800b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:37.456507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:37.457957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:37.476539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8a54b-671c-4251-85a4-4d1bb296e669 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:37.503004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:37.533837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:37.534731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:37.535015 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:37.535290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] PATH_INFO: `/v3/projects/1dedf3249e334c8bb12b1fa29af7c07f` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:37.536565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dedf3249e334c8bb12b1fa29af7c07f)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:37.538493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:37.643942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dedf3249e334c8bb12b1fa29af7c07f', 'request_id': 'req-3952fbc9-137a-479b-8e81-e209d21ef875'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.717585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dedf3249e334c8bb12b1fa29af7c07f', 'request_id': 'req-3952fbc9-137a-479b-8e81-e209d21ef875'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.724287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dedf3249e334c8bb12b1fa29af7c07f', 'request_id': 'req-3952fbc9-137a-479b-8e81-e209d21ef875'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.738216 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] The token cache is being invalidate because project 1dedf3249e334c8bb12b1fa29af7c07f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:37.738411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.739253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-3952fbc9-137a-479b-8e81-e209d21ef875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:37.741050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1258/2540] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:37 2022] DELETE /identity/v3/projects/1dedf3249e334c8bb12b1fa29af7c07f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:37.745735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:37.751977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:37.751977 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:37.768778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:37.770066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:37.842198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1283/2541] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:37.850376 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:37.850668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:37.850996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:37.870419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:37.871649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:37.880189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1259/2542] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:37.884500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:37.894422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a0f70-7c18-4102-937e-64c4848f0399 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:37.899161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:37.899869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:37.900153 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:37.900430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:37.901158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:36:37.901465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:37.903066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:37.916230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1284/2543] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:36:37.920487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:37.930259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85f26d-1437-40e8-932b-6a789128ca59 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:37.934916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:37.935611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:37.935897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:37.936178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:37.937547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:37.939481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.021303 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1260/2544] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:37 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:38.025411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.036290 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2afcb-034b-4910-b91b-2e3976c7a04a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.065033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:38.092332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.093004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.093910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.094191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.094889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.096940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.236972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1285/2545] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:38 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:38.244618 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.262654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eccf21-c5dd-413a-8047-4c860e762ac4 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.267912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.268649 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.268946 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.269224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.269951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.272411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.284317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1261/2546] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:38.288397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.298964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c86295-c7dd-4035-ae43-822d6e5eaa7f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.304315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a70f9-3d77-4ad4-a71f-f9eb0cd6a856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.305022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a70f9-3d77-4ad4-a71f-f9eb0cd6a856 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.305443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a70f9-3d77-4ad4-a71f-f9eb0cd6a856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.305728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a70f9-3d77-4ad4-a71f-f9eb0cd6a856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.306457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a70f9-3d77-4ad4-a71f-f9eb0cd6a856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.308157 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a70f9-3d77-4ad4-a71f-f9eb0cd6a856 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.319357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1286/2547] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:38.323290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.333556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a1acf-2a13-42b4-8f25-c8c25f70a072 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.338055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.338930 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.339213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.339523 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] PATH_INFO: `/v3/projects/7f934f41ddad49ddad62e6081c03f5c8/users/69be6959ce4a474d99dfb421a63d7972/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.354113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f934f41ddad49ddad62e6081c03f5c8, user_id=69be6959ce4a474d99dfb421a63d7972, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.356805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.424789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1262/2548] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:38 2022] PUT /identity/v3/projects/7f934f41ddad49ddad62e6081c03f5c8/users/69be6959ce4a474d99dfb421a63d7972/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:38.435091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.435453 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.435739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.463292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] MFA Rules not processed for user `69be6959ce4a474d99dfb421a63d7972`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:38.464558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:38.528907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1287/2549] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:36:38.533463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.590922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:38.594928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989612d3-a48d-4518-aa30-a0dab8967398 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.599895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.602282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.602282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.602282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.602282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.603804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.679806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1263/2550] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:38 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:38.683790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.695195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696e51-41a0-445a-999a-9f2def6b1d34 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.751624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.752300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.752558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.752832 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.753513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.755446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.826600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1288/2551] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:38 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:38.830500 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.840647 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a4b0-475c-4a3a-b63f-788a63d4943a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.846347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.847012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.847316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.847623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.848345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.849968 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.868924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1264/2552] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:38.871254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.881664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217c63d-235c-489c-bc49-363dee82415d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.886513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a9651-6957-459e-b9d7-5c7ab4be7bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.887253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a9651-6957-459e-b9d7-5c7ab4be7bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.887651 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a9651-6957-459e-b9d7-5c7ab4be7bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.887942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a9651-6957-459e-b9d7-5c7ab4be7bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.888808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a9651-6957-459e-b9d7-5c7ab4be7bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.890526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a9651-6957-459e-b9d7-5c7ab4be7bbf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:38.901155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1289/2553] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:36:38.906002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:38.916150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08204d99-75e1-46ef-bead-8e02d79c0ffe admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:38.922815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:38.923749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:38.924029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:38.924304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] PATH_INFO: `/v3/projects/040714a214e34a8d9ea591a449e486a6/users/0eeadea4efc04360bed1956bf2995578/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:38.939848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040714a214e34a8d9ea591a449e486a6, user_id=0eeadea4efc04360bed1956bf2995578, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:38.942466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.036646 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1265/2554] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:38 2022] PUT /identity/v3/projects/040714a214e34a8d9ea591a449e486a6/users/0eeadea4efc04360bed1956bf2995578/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:39.106403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.106602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.106871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.133523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] MFA Rules not processed for user `0eeadea4efc04360bed1956bf2995578`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:39.134716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:39.139614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:39.198321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1290/2555] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:39.203546 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.264703 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3d77-5e0d-4412-8d53-eed850582359 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:39.269426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:39.270087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.270375 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.270653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.271383 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:39.273283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.368032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1266/2556] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:39 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:36:39.369940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.385727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819d73-604c-4365-b0c0-67d0bded3249 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:39.451558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:39.452240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.453061 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.453450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.454143 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:39.456089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.546684 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1291/2557] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:39 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:39.550666 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.561014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecbf48-2e1f-42fe-9f2f-27170858efec admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:39.565750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:39.566440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.566729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.567005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.567767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:39.569355 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.580478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1267/2558] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:39.584622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.595649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66caf57-eeeb-46a1-89d0-f6b731066adf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:39.600373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:39.601284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.601573 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.601852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] PATH_INFO: `/v3/projects/295d0c9a4ebc435fbc90ebb366bdf0dc/users/aade9520d35f484eb1cd7278c9c28b9a/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.616550 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295d0c9a4ebc435fbc90ebb366bdf0dc, user_id=aade9520d35f484eb1cd7278c9c28b9a, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:39.619292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.683734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1292/2559] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:39 2022] PUT /identity/v3/projects/295d0c9a4ebc435fbc90ebb366bdf0dc/users/aade9520d35f484eb1cd7278c9c28b9a/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:39.688354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.698308 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7de3d-73fd-47fa-9b17-5dcb4c171390 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:39.760682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd3922-e682-4557-abb9-7e05fb93b745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:39.761358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd3922-e682-4557-abb9-7e05fb93b745 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.761645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd3922-e682-4557-abb9-7e05fb93b745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.761929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd3922-e682-4557-abb9-7e05fb93b745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.762657 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd3922-e682-4557-abb9-7e05fb93b745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:39.764288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd3922-e682-4557-abb9-7e05fb93b745 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.773906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1268/2560] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:39.778038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.788828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef5e3e-2868-4d0e-a1f2-37b0fd809115 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:39.793496 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:39.794372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.794663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.794946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] PATH_INFO: `/v3/projects/295d0c9a4ebc435fbc90ebb366bdf0dc/users/aade9520d35f484eb1cd7278c9c28b9a/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.797625 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295d0c9a4ebc435fbc90ebb366bdf0dc, user_id=aade9520d35f484eb1cd7278c9c28b9a, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:39.800295 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:39.810809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1293/2561] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:39 2022] PUT /identity/v3/projects/295d0c9a4ebc435fbc90ebb366bdf0dc/users/aade9520d35f484eb1cd7278c9c28b9a/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:39.821285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:39.821581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:39.821867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:39.850103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] MFA Rules not processed for user `aade9520d35f484eb1cd7278c9c28b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:39.851535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:39.856640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:39.923614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1269/2562] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:39.934779 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:39.938627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:40.014321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc5f4e-8998-4977-807e-88fd5a3d3402 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:40.018571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:40.019253 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:40.019604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:40.019889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:40.029269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:40.030863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:40.041469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1294/2563] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:40.239582 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:40.240963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:40.253451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337c3fc-7c0b-4991-aaed-98c1e0548e82 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:40.257063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd6142-94a2-4b14-ae05-b3561ecdb6c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:40.257717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd6142-94a2-4b14-ae05-b3561ecdb6c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:40.258004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd6142-94a2-4b14-ae05-b3561ecdb6c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:40.258285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd6142-94a2-4b14-ae05-b3561ecdb6c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:40.316127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd6142-94a2-4b14-ae05-b3561ecdb6c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:40.317781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd6142-94a2-4b14-ae05-b3561ecdb6c3 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:40.327034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1270/2564] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:45.606221 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:45.610898 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:45.671989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107c552-7332-473f-b86b-8765a1162a25 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:45.676664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:45.677542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:45.677827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:45.678105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] PATH_INFO: `/v3/users/69be6959ce4a474d99dfb421a63d7972` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:45.679296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69be6959ce4a474d99dfb421a63d7972)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:45.681278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:45.881620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69be6959ce4a474d99dfb421a63d7972'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:45.882193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69be6959ce4a474d99dfb421a63d7972'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:45.894833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69be6959ce4a474d99dfb421a63d7972'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:45.901741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69be6959ce4a474d99dfb421a63d7972'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:45.973383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69be6959ce4a474d99dfb421a63d7972'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:45.981643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1295/2565] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:45 2022] DELETE /identity/v3/users/69be6959ce4a474d99dfb421a63d7972 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:45.986700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:45.987435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:45.987435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:46.006380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:46.007993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:46.078944 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1271/2566] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:46.088860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:46.090326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:46.153118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36780-bc2a-4ca0-bb50-aad69d6fde52 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:46.157529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:46.158183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:46.158468 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:46.158749 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:46.168255 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:46.180818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:46.180818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1296/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:46.599325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:46.601779 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:46.616879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a89e0-d972-4eed-a7bb-937324f44b98 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:46.631550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:46.631711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:46.631949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:46.632365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] PATH_INFO: `/v3/projects/7f934f41ddad49ddad62e6081c03f5c8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:46.633721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f934f41ddad49ddad62e6081c03f5c8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:46.636257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:46.738683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:46.747285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f934f41ddad49ddad62e6081c03f5c8', 'request_id': 'req-7b2706f2-f7f5-4a48-be2f-518ceff2f061'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:46.820185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f934f41ddad49ddad62e6081c03f5c8', 'request_id': 'req-7b2706f2-f7f5-4a48-be2f-518ceff2f061'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:46.826286 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f934f41ddad49ddad62e6081c03f5c8', 'request_id': 'req-7b2706f2-f7f5-4a48-be2f-518ceff2f061'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:46.838139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] The token cache is being invalidate because project 7f934f41ddad49ddad62e6081c03f5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:46.838427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:46.839081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-7b2706f2-f7f5-4a48-be2f-518ceff2f061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:46.840664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1272/2568] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:46 2022] DELETE /identity/v3/projects/7f934f41ddad49ddad62e6081c03f5c8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:46.846025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:46.855743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64383de-9899-4e50-bf64-c3025d481f34 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:46.876165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:46.925806 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:46.926600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:46.926882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:46.927215 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] PATH_INFO: `/v3/users/0eeadea4efc04360bed1956bf2995578` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:46.933337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eeadea4efc04360bed1956bf2995578)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:46.933337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:47.102006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eeadea4efc04360bed1956bf2995578'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:47.102662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eeadea4efc04360bed1956bf2995578'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:47.114718 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eeadea4efc04360bed1956bf2995578'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:47.121259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eeadea4efc04360bed1956bf2995578'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:47.213354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eeadea4efc04360bed1956bf2995578'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:47.221287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1297/2569] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:46 2022] DELETE /identity/v3/users/0eeadea4efc04360bed1956bf2995578 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:36:47.646945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:47.647461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:47.647563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:47.665138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:47.666334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:47.725175 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:47.725748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1273/2570] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:36:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:47.726874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:47.798159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:47.799728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:47.812758 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564660e-6615-411d-999e-bec21a716a19 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:47.819291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:47.820206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:47.820495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:47.820778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] PATH_INFO: `/v3/projects/040714a214e34a8d9ea591a449e486a6` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:47.822483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=040714a214e34a8d9ea591a449e486a6)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:47.825329 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:47.878414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627581-2223-4137-88f9-a48ecefa75d2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:47.887418 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:47.888075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:47.888415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:47.888701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:47.897411 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:47.899050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:47.908704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1274/2571] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:36:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 01 04:36:47.926124 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:47.929297 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040714a214e34a8d9ea591a449e486a6', 'request_id': 'req-26780f25-208e-47b4-8f7a-9140da2ebf72'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:47.935533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040714a214e34a8d9ea591a449e486a6', 'request_id': 'req-26780f25-208e-47b4-8f7a-9140da2ebf72'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.033518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040714a214e34a8d9ea591a449e486a6', 'request_id': 'req-26780f25-208e-47b4-8f7a-9140da2ebf72'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.063604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] The token cache is being invalidate because project 040714a214e34a8d9ea591a449e486a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:48.066154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.066154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.070585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1298/2572] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:47 2022] DELETE /identity/v3/projects/040714a214e34a8d9ea591a449e486a6 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:48.074790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:48.087700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0383e-2f98-4421-9a0c-7e336be7c639 service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:48.164266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:48.165316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:48.165605 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:48.166059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] PATH_INFO: `/v3/users/aade9520d35f484eb1cd7278c9c28b9a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:48.167358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aade9520d35f484eb1cd7278c9c28b9a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:48.169297 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:48.170991 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:48.349516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aade9520d35f484eb1cd7278c9c28b9a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.363943 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aade9520d35f484eb1cd7278c9c28b9a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.364923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aade9520d35f484eb1cd7278c9c28b9a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.393873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aade9520d35f484eb1cd7278c9c28b9a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.478040 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-82dfe9a6-59ae-405f-8f2b-6c56b2487615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aade9520d35f484eb1cd7278c9c28b9a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:48.486366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1275/2573] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:48 2022] DELETE /identity/v3/users/aade9520d35f484eb1cd7278c9c28b9a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:49.183466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:49.187517 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:49.227133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26780f25-208e-47b4-8f7a-9140da2ebf72 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:49.334524 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:49.335514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:49.335803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:49.336083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] PATH_INFO: `/v3/projects/295d0c9a4ebc435fbc90ebb366bdf0dc` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:49.337987 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=295d0c9a4ebc435fbc90ebb366bdf0dc)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:49.340125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:49.484930 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '295d0c9a4ebc435fbc90ebb366bdf0dc', 'request_id': 'req-3d78602c-bb8e-4175-9f19-e100ad644b66'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:49.496954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '295d0c9a4ebc435fbc90ebb366bdf0dc', 'request_id': 'req-3d78602c-bb8e-4175-9f19-e100ad644b66'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:49.598361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '295d0c9a4ebc435fbc90ebb366bdf0dc', 'request_id': 'req-3d78602c-bb8e-4175-9f19-e100ad644b66'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:49.617278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] The token cache is being invalidate because project 295d0c9a4ebc435fbc90ebb366bdf0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:49.617772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:49.619284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d78602c-bb8e-4175-9f19-e100ad644b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:49.629249 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1299/2574] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:49 2022] DELETE /identity/v3/projects/295d0c9a4ebc435fbc90ebb366bdf0dc => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:49.630190 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:49.630817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:49.630935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:49.657815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:49.659101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:49.758393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1276/2575] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:49.766915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:49.767599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:49.768044 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:49.819842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:49.821719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:49.851963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1300/2576] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:49.855964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:49.869161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40405f-1a4a-4ceb-8081-dfea090563ff None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:49.875444 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:49.876534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:49.877199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:49.877650 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:49.878722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:36:49.879394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:49.881602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:49.905512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1277/2577] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:36:49.909830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:49.937973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc4aa-5a33-4d99-a314-2ba7846a5127 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:49.950729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:49.951460 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:49.951750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:49.952032 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:49.954464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:49.957725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:50.109766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1301/2578] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:36:49 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:36:50.114305 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:50.141839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122542af-2299-44ec-a3c7-a901e26d72ac admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:50.258095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:50.258821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:50.259097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:50.259402 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:50.260354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:50.262344 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:50.277704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:50.374768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1278/2579] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:36:50 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:36:50.376715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:50.397585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490680-5f78-4196-969e-409f388f733e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:50.419538 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:50.425726 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:50.426130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:50.430494 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:50.431368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:50.433090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:50.473980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1302/2580] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:36:50.476325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:50.489283 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2451f2f-4f4e-43a9-aae5-6e2ece1796ae admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:50.497324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f15c2e-c802-4fe2-b3f4-a93c4edd1eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:50.498685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f15c2e-c802-4fe2-b3f4-a93c4edd1eec admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:50.499263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f15c2e-c802-4fe2-b3f4-a93c4edd1eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:50.499858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f15c2e-c802-4fe2-b3f4-a93c4edd1eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:50.501690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f15c2e-c802-4fe2-b3f4-a93c4edd1eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:50.504339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f15c2e-c802-4fe2-b3f4-a93c4edd1eec admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:50.524615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1279/2581] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:36:50.526070 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:50.537828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f07f5a-5e08-472c-98f0-b68cddc37387 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:50.543695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:50.544978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:50.545485 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:50.546437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] PATH_INFO: `/v3/projects/b3fd0d00e9794a499c258ebdcae307ad/users/50512ed3d2934880ba74bac4967eb12e/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:50.572935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fd0d00e9794a499c258ebdcae307ad, user_id=50512ed3d2934880ba74bac4967eb12e, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:50.575665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:50.651773 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:50.667358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1303/2582] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:36:50 2022] PUT /identity/v3/projects/b3fd0d00e9794a499c258ebdcae307ad/users/50512ed3d2934880ba74bac4967eb12e/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:50.673981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:50.674533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:50.674533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:50.709034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] MFA Rules not processed for user `50512ed3d2934880ba74bac4967eb12e`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:50.710300 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:50.835218 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1280/2583] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:50.848789 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:50.850519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:50.926637 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296284d-2198-4df2-8d66-bfd5152d2aae admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:50.930807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1025-2d04-4419-a89d-ee30814a7750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:50.931696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1025-2d04-4419-a89d-ee30814a7750 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:50.932325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1025-2d04-4419-a89d-ee30814a7750 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:50.932771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1025-2d04-4419-a89d-ee30814a7750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:50.943828 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1025-2d04-4419-a89d-ee30814a7750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:50.946810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1025-2d04-4419-a89d-ee30814a7750 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:50.962547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1304/2584] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:50.995525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:51.080304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648d784-840f-48fa-a730-5bb8a9904108 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:51.087764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:51.089084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:51.089543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:51.089840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] PATH_INFO: `/v3/users/50512ed3d2934880ba74bac4967eb12e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:51.091101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=50512ed3d2934880ba74bac4967eb12e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:51.093765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:51.313794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50512ed3d2934880ba74bac4967eb12e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:51.324691 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50512ed3d2934880ba74bac4967eb12e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:51.325730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50512ed3d2934880ba74bac4967eb12e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:51.352810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50512ed3d2934880ba74bac4967eb12e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:51.358078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:36:51.449278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50512ed3d2934880ba74bac4967eb12e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:51.457245 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1281/2585] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:36:50 2022] DELETE /identity/v3/users/50512ed3d2934880ba74bac4967eb12e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:36:51.464312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:51.464312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:51.464312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:51.481081 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:36:51.482315 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:51.556246 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1305/2586] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:36:51.568211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:51.569659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:51.631685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e3a44-c467-4160-ab43-cefb4a8b67cf admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:51.635358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:51.636008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:51.636287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:51.636561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:51.646580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:51.648057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:51.657675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1282/2587] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:36:52.305612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:36:52.305612 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:36:52.332480 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a838a-289a-4d9b-b4c6-28017b057ae3 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:36:52.355530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:36:52.357971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:36:52.358361 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:36:52.358969 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] PATH_INFO: `/v3/projects/b3fd0d00e9794a499c258ebdcae307ad` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:36:52.360434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fd0d00e9794a499c258ebdcae307ad)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:36:52.362665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:36:52.525862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fd0d00e9794a499c258ebdcae307ad', 'request_id': 'req-67654a9e-581c-4a84-9338-68dea5204833'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:52.534085 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fd0d00e9794a499c258ebdcae307ad', 'request_id': 'req-67654a9e-581c-4a84-9338-68dea5204833'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:52.621011 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fd0d00e9794a499c258ebdcae307ad', 'request_id': 'req-67654a9e-581c-4a84-9338-68dea5204833'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:52.639075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] The token cache is being invalidate because project b3fd0d00e9794a499c258ebdcae307ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:36:52.639568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:52.641235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:36:52.659333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1306/2588] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:36:52 2022] DELETE /identity/v3/projects/b3fd0d00e9794a499c258ebdcae307ad => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:37:01.940026 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:02.635752 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:06.665357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:06.666837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:06.731782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff38d6-4db7-4f9c-b2e2-f1b2d53beab0 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:06.776611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e0959756bd54aa58afb14b9b45d29ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:06.777253 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:06.777533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:06.777805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:06.779735 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:06.847906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:06.849255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:06.860764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1283/2589] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:37:07.068492 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:07.070002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:07.127663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654a9e-581c-4a84-9338-68dea5204833 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:07.131138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:07.131940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:07.132222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:07.132534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:07.134261 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:07.144767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:07.146213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07302c-acb3-4d48-8af1-82004d2b035c service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:07.154023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1307/2590] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:37:07.227736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:07.237396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9176521-b74e-4b3d-aa38-081631450ead service glance] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:07.240947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:07.241616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:07.241896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:07.242173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:07.243940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:07.257290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:07.258752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:07.269838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1284/2591] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:37:09.895974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:09.896445 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:09.896549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:09.914303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:37:09.920063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:09.971222 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1308/2592] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:37:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:37:09.984133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:09.986106 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:10.033791 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:10.056225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca7d5-b36c-4d44-8deb-549705ef9778 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:10.061429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49648f07-ab14-4fd6-9f06-0b539c7ed944 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:10.062103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648f07-ab14-4fd6-9f06-0b539c7ed944 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:10.062385 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648f07-ab14-4fd6-9f06-0b539c7ed944 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:10.062673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648f07-ab14-4fd6-9f06-0b539c7ed944 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:10.071178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49648f07-ab14-4fd6-9f06-0b539c7ed944 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:10.072836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49648f07-ab14-4fd6-9f06-0b539c7ed944 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:10.081749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1285/2593] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:37:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:37:19.964387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=51 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:20.094183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=12 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:30.455753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:30.457931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:30.469366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14cc6d-8151-4506-8c57-54a1636c9688 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:30.472873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:30.473503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:30.473786 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:30.474063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:30.475788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:30.488868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:30.490289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:30.501019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1309/2594] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:37:30.919164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:30.919758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:30.919758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:30.937998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:37:30.939210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:30.947109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1286/2595] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jun 1 04:37:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:37:30.959062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:30.970513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cad0c9-9572-4e9c-90fd-b897f2ce9519 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:30.975876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:30.976509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:30.976793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:30.977068 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:30.985563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:30.987127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:30.995600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1310/2596] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:37:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:37:31.679091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:31.690172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98679486-f0f0-4321-a4a4-d4bb67d4f9e2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:31.695021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:31.695709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:31.696197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:31.696688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:31.699588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:31.716173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:31.720252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590641ff-d905-485c-a550-1470d50a8409 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:31.731310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1287/2597] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:37:40.992379 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=12 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:41.725042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=9 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:43.062842 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:43.064242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:43.130252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7520ca-735e-4d08-9f83-270837dc844b service swift] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:43.133881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:43.134527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:43.134813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:43.135090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:43.136903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:43.195795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:43.197250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:43.206243 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1311/2598] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:37:53.202103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=48 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:37:54.775768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:54.776285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:54.776394 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:54.794552 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] MFA Rules not processed for user `c052b2f518064843928b4eff569e913f`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:37:54.795797 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:54.803522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1288/2599] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:37:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:37:58.691875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:37:58.700812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a64847-f623-40dd-9c60-e01825ddcbc8 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:37:58.704596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:37:58.705239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:37:58.705530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:37:58.705811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:37:58.707710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:37:58.717947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:37:58.719432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:37:58.727654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1312/2600] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:37:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:38:04.788836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:08.724535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=6 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:14.652861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:14.662619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538249e-8333-4f26-b84b-fab5d9fe72e7 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:14.666294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:14.666936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:14.667272 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:14.667563 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:14.669540 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:14.733767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:14.743222 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:14.746987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1289/2601] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:38:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:38:24.741856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=30 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:40.190874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:40.202213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281829fe-c93e-4f3b-951a-3ff0542b90c3 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:40.205916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:40.206560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:40.206847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:40.207127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:40.209000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:40.290943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:40.292500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:40.312242 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1313/2602] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:38:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:38:44.518867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:44.519466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:44.550640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb54af5-f55e-41eb-b7d2-6a26376a110c service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:44.560100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:44.560785 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:44.564722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:44.565198 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:44.568356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:44.611736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:44.614365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:44.642185 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1290/2603] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:38:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:38:50.309429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=41 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:51.641847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:51.650118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e672cd-1d63-4689-b7f3-ad8f4cf33a8f service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:51.654807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:51.655489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:51.655775 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:51.656084 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:51.657847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:51.722770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:51.724260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:51.735155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1314/2604] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:38:54.634611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=6 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:56.470459 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:56.472282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:56.488621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e8075-b1b4-45bc-996c-96e4b304de81 service nova] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:56.493541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:56.494519 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:56.495107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:56.495240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] PATH_INFO: `/v3/users/85df2f6595d3489bb5d1d547cb2cd900` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:56.496555 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=85df2f6595d3489bb5d1d547cb2cd900)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:56.498565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:56.649158 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85df2f6595d3489bb5d1d547cb2cd900'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:56.658417 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85df2f6595d3489bb5d1d547cb2cd900'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:56.659128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85df2f6595d3489bb5d1d547cb2cd900'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:56.672134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85df2f6595d3489bb5d1d547cb2cd900'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:56.730890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c54a1f81-e1ef-4911-97ea-fd90fac57fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85df2f6595d3489bb5d1d547cb2cd900'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:56.738279 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1291/2605] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:38:56 2022] DELETE /identity/v3/users/85df2f6595d3489bb5d1d547cb2cd900 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:38:56.974345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:56.985564 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59da1d-6a59-4c61-a586-b8321640fe1b service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.045304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.046173 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.046458 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.046732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] PATH_INFO: `/v3/projects/d12a9b5dc38b4aa7aed77adc89d0e6a4` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.047940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12a9b5dc38b4aa7aed77adc89d0e6a4)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.049891 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.179715 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12a9b5dc38b4aa7aed77adc89d0e6a4', 'request_id': 'req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:57.185865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12a9b5dc38b4aa7aed77adc89d0e6a4', 'request_id': 'req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:57.280576 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12a9b5dc38b4aa7aed77adc89d0e6a4', 'request_id': 'req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:57.308621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] The token cache is being invalidate because project d12a9b5dc38b4aa7aed77adc89d0e6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:38:57.308621 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:57.314147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d8a2e7a5-d2fd-47b3-acc4-00f9102f5738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:38:57.314147 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1315/2606] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:38:56 2022] DELETE /identity/v3/projects/d12a9b5dc38b4aa7aed77adc89d0e6a4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:38:57.317980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083690ca-98fd-42f2-b911-b1d511c29e92 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.318296 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083690ca-98fd-42f2-b911-b1d511c29e92 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.318626 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083690ca-98fd-42f2-b911-b1d511c29e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.340592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-083690ca-98fd-42f2-b911-b1d511c29e92 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:38:57.341853 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-083690ca-98fd-42f2-b911-b1d511c29e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:57.418642 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1292/2607] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:38:57.418857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.418857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.419090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.425021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.425668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.425668 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.441942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:38:57.443488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:57.444503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:38:57.445680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:57.455822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1293/2608] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:38:57.459954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.466339 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:57.469281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9db2b-0e4b-4b11-937f-2bb202277332 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.473938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.474721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.475010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.475503 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.476240 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:38:57.476548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.478990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.491848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1294/2609] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:38:57.507695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.514677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1316/2610] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jun 1 04:38:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:38:57.518955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde21234-4eba-4e70-b617-bd35fa80f8f7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.523950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.527416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.527416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.527416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.527416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.528994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.646146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1295/2611] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:38:57 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:38:57.650159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.662027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2f02c-640f-4c51-bceb-b2bf75f2e5c0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.722031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.722722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.723066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.723178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.723532 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.724189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.724812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:57.726136 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.763466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:57.799488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4ced0-56e8-4e21-8883-2de0b20a378a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.804767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.805424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.805717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.806022 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.837072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1317/2612] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:38:57 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:38:57.841015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.851911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7ee4f-b347-471c-afd0-200e607de0bd admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.858440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.859111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.859447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.859802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.860603 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.862568 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.866414 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.868073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.874934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1318/2613] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:38:57.878354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.881435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1296/2614] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 01 04:38:57.894266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4cf4-5f04-469f-b7dc-5e8bc7ea2761 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.903294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752dbbc-7ceb-45d9-be76-e9a90bccf835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.904016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dbbc-7ceb-45d9-be76-e9a90bccf835 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.904306 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dbbc-7ceb-45d9-be76-e9a90bccf835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.904584 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dbbc-7ceb-45d9-be76-e9a90bccf835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.905321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752dbbc-7ceb-45d9-be76-e9a90bccf835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.906946 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752dbbc-7ceb-45d9-be76-e9a90bccf835 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:57.919354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1319/2615] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:38:57.922835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:57.935342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde7b2d-084e-45df-968b-7de5e5628f96 service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:57.940028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:57.940962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:57.941434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:57.941871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] PATH_INFO: `/v3/projects/40c58fd7462e4e99938c6f7e1c14022b/users/162ede21b33246b3a2931be3377f5753/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:57.959191 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c58fd7462e4e99938c6f7e1c14022b, user_id=162ede21b33246b3a2931be3377f5753, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:57.961860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.067669 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1297/2616] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:38:57 2022] PUT /identity/v3/projects/40c58fd7462e4e99938c6f7e1c14022b/users/162ede21b33246b3a2931be3377f5753/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:38:58.078851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.079435 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.079533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.112107 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] MFA Rules not processed for user `162ede21b33246b3a2931be3377f5753`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:38:58.113507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:58.186440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1320/2617] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:38:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:38:58.190887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.256192 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa53759-5b99-46c7-b7a9-17b3d43a6b65 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:58.261304 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:58.261980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.262267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.262549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.263264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:58.265361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.350936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1298/2618] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:38:58 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:38:58.356337 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.372847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e076bcc-1c80-42aa-bfb1-cb68e36f0223 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:58.403373 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:58.437837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:58.438522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.438813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.439097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.439841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:58.441883 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.549150 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1321/2619] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:38:58 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:38:58.553165 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.563682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df036a7-ea63-49ff-8380-f0452353a64a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:58.568397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:58.569058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.569347 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.569625 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.570356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:58.572008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.583017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1299/2620] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:38:58.588601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.598659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03972941-a27a-4a6c-84f8-a93dfa7903ab admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:58.603387 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:58.604318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.604604 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.604884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] PATH_INFO: `/v3/projects/db707fa900ec476abf8fb4055c7bcd78/users/9eea3bd77ac04795b9718e23722f41eb/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.620198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db707fa900ec476abf8fb4055c7bcd78, user_id=9eea3bd77ac04795b9718e23722f41eb, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:58.622955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.721057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1322/2621] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:38:58 2022] PUT /identity/v3/projects/db707fa900ec476abf8fb4055c7bcd78/users/9eea3bd77ac04795b9718e23722f41eb/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:38:58.725380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.735495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf24cf8-a628-49ad-8fd2-4f3722d0144f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:58.756200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:38:58.794751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7bc25-ecf1-44ad-8b84-1e5e7a3ad106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:58.795502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7bc25-ecf1-44ad-8b84-1e5e7a3ad106 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.795786 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7bc25-ecf1-44ad-8b84-1e5e7a3ad106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.796063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7bc25-ecf1-44ad-8b84-1e5e7a3ad106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.796787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7bc25-ecf1-44ad-8b84-1e5e7a3ad106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:58.798388 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7bc25-ecf1-44ad-8b84-1e5e7a3ad106 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.809128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1300/2622] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:38:58.813624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.824611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d9ed-6bf1-4d0a-b4d0-d7950cbe04f0 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:58.829236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:58.830111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.830397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.830677 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] PATH_INFO: `/v3/projects/db707fa900ec476abf8fb4055c7bcd78/users/9eea3bd77ac04795b9718e23722f41eb/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.832670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db707fa900ec476abf8fb4055c7bcd78, user_id=9eea3bd77ac04795b9718e23722f41eb, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:58.835292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:58.846478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1323/2623] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:38:58 2022] PUT /identity/v3/projects/db707fa900ec476abf8fb4055c7bcd78/users/9eea3bd77ac04795b9718e23722f41eb/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:38:58.858136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:58.858435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:58.858724 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:58.887053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] MFA Rules not processed for user `9eea3bd77ac04795b9718e23722f41eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:38:58.888536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:58.960049 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1301/2624] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:38:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:38:58.971077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:38:58.972843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:38:59.040957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cacd4-7623-4f0f-b649-86a4bbeda32a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:38:59.044860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:38:59.045501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:38:59.045792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:38:59.046072 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:38:59.055952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:38:59.058082 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:38:59.068191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1324/2625] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:38:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:02.219405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:02.221034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:02.298048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6529f3-a8f9-4e31-b341-50477c1b1512 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:02.310547 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:02.311769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:02.312153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:02.312386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:02.324567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:02.345350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:02.430627 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:02.434776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:02.436585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e12ea-1138-4696-9487-5ace4ef5d261 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:02.447905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032a9c-2ff7-46d8-9bce-55f026d9d947 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:02.456001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:02.456348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1302/2626] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:02.456912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:02.457201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:02.457482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] PATH_INFO: `/v3/users/162ede21b33246b3a2931be3377f5753` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:02.460263 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162ede21b33246b3a2931be3377f5753)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:02.462504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:02.500919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:02.669060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162ede21b33246b3a2931be3377f5753'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:02.670587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162ede21b33246b3a2931be3377f5753'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:02.690654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162ede21b33246b3a2931be3377f5753'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:02.701120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162ede21b33246b3a2931be3377f5753'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:02.766721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162ede21b33246b3a2931be3377f5753'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:02.780013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1325/2627] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:02 2022] DELETE /identity/v3/users/162ede21b33246b3a2931be3377f5753 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:02.784338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:02.784629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:02.784905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:02.804922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:02.809841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:02.929891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1303/2628] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:02.940325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:02.941889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:03.013702 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfe009-193e-45f5-889b-04c398706d87 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:03.019809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:03.020484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:03.020769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:03.021050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:03.033213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:03.034750 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:03.046753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1326/2629] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:04.105808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:04.107679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:04.121291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7801590-2e5a-419b-9c2f-f8ac82916ebf None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:04.131908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:04.134171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:04.134311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:04.135197 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] PATH_INFO: `/v3/projects/40c58fd7462e4e99938c6f7e1c14022b` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:04.137111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c58fd7462e4e99938c6f7e1c14022b)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:04.139379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:04.253521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c58fd7462e4e99938c6f7e1c14022b', 'request_id': 'req-4c76d248-571a-459b-b3d9-6f157db74043'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.302996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c58fd7462e4e99938c6f7e1c14022b', 'request_id': 'req-4c76d248-571a-459b-b3d9-6f157db74043'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.311322 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c58fd7462e4e99938c6f7e1c14022b', 'request_id': 'req-4c76d248-571a-459b-b3d9-6f157db74043'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.341146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] The token cache is being invalidate because project 40c58fd7462e4e99938c6f7e1c14022b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:04.343679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.351755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.353472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1304/2630] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:04 2022] DELETE /identity/v3/projects/40c58fd7462e4e99938c6f7e1c14022b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:04.359179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:04.378628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be93c-7815-4224-8dd1-2e351c5717c6 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:04.467469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:04.468535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:04.468881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:04.469168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] PATH_INFO: `/v3/users/9eea3bd77ac04795b9718e23722f41eb` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:04.470613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eea3bd77ac04795b9718e23722f41eb)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:04.472695 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:04.717793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:04.719663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eea3bd77ac04795b9718e23722f41eb'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.720344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eea3bd77ac04795b9718e23722f41eb'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.735171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eea3bd77ac04795b9718e23722f41eb'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.743993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eea3bd77ac04795b9718e23722f41eb'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.815933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d3d43cf2-59b4-4891-bed5-111581db373f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eea3bd77ac04795b9718e23722f41eb'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:04.825837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1327/2631] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:04 2022] DELETE /identity/v3/users/9eea3bd77ac04795b9718e23722f41eb => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:05.569640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:05.574341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:05.595856 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76d248-571a-459b-b3d9-6f157db74043 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:05.674017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:05.675033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:05.675351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:05.675633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] PATH_INFO: `/v3/projects/db707fa900ec476abf8fb4055c7bcd78` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:05.677598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db707fa900ec476abf8fb4055c7bcd78)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:05.679181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:05.707037 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:05.807249 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db707fa900ec476abf8fb4055c7bcd78', 'request_id': 'req-ab861f3d-47c4-45b7-bddd-062f892c9ea4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:05.892825 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db707fa900ec476abf8fb4055c7bcd78', 'request_id': 'req-ab861f3d-47c4-45b7-bddd-062f892c9ea4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:05.909497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db707fa900ec476abf8fb4055c7bcd78', 'request_id': 'req-ab861f3d-47c4-45b7-bddd-062f892c9ea4'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:05.927226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] The token cache is being invalidate because project db707fa900ec476abf8fb4055c7bcd78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:05.928250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:05.929127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ab861f3d-47c4-45b7-bddd-062f892c9ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:05.947578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1305/2632] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:05 2022] DELETE /identity/v3/projects/db707fa900ec476abf8fb4055c7bcd78 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:05.949487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:05.949941 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:05.950185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:05.971556 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:05.973370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:06.061848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.062145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.068381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.068700 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1328/2633] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:06.084860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:06.086166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:06.099277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1306/2634] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:39:06.104354 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:06.117649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd93aa-da36-46de-8d5f-138fdc18b811 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:06.132599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:06.133083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.137472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.137472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.137472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:39:06.137472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:06.137472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:06.162983 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1329/2635] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:39:06.166199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:06.179664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a32c05-51f0-403b-982a-f945a5085472 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:06.184601 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:06.185278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.185565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.185846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.186571 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:06.188548 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:06.319273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1307/2636] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:06 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:06.320948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:06.332123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9580-3b92-4647-88fc-3beef6ded714 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:06.415522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:06.416034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.416429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.416994 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.418961 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:06.421689 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:06.572976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1330/2637] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:06 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:06.576948 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:06.587813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66feafb-7586-42e2-85ce-bfb7d595996c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:06.593248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:06.594714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.594714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.594714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.595969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:06.598119 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:06.612156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1308/2638] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:06.616611 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:06.636761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18444a-955f-42fc-9d01-b73a63244bc8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:06.650530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3bfe3-8cb7-4ae0-9b3b-eb9d0a418043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:06.650530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3bfe3-8cb7-4ae0-9b3b-eb9d0a418043 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.650530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3bfe3-8cb7-4ae0-9b3b-eb9d0a418043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.650530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3bfe3-8cb7-4ae0-9b3b-eb9d0a418043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.651123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3bfe3-8cb7-4ae0-9b3b-eb9d0a418043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:06.651123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3bfe3-8cb7-4ae0-9b3b-eb9d0a418043 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:06.665426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1331/2639] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:06.669672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:06.680298 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74931ada-72d1-464f-9342-f1368bbed902 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:06.685741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:06.688127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.688127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.688127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] PATH_INFO: `/v3/projects/f0430d8dc58d48f5a74831c36438a98e/users/bcf9566afb2a4e4ea04f6a4705ada3f9/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.704021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0430d8dc58d48f5a74831c36438a98e, user_id=bcf9566afb2a4e4ea04f6a4705ada3f9, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:06.706895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:06.776345 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1309/2640] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:06 2022] PUT /identity/v3/projects/f0430d8dc58d48f5a74831c36438a98e/users/bcf9566afb2a4e4ea04f6a4705ada3f9/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:06.789807 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:06.790104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:06.790562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:06.865566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] MFA Rules not processed for user `bcf9566afb2a4e4ea04f6a4705ada3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:06.882151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:06.918412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:07.006448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1332/2641] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:07.007511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.008227 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.010319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:07.094581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108b64e-8060-41c7-815e-53ad7a2d7b2d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.101429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.102690 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.103267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.104302 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] PATH_INFO: `/v3/users/a0a147daff9846e699c26259fac4e004` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.106044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a147daff9846e699c26259fac4e004)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.107489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a94591-ec5a-4542-a937-b79e71cdcd30 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.108858 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.113762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.114503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.114796 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.115079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.115834 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.118306 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.290640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1333/2642] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:07 2022] POST /identity/v3/projects => generated 379 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:07.292736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.363287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c411f-13dc-410d-936d-4ed2f152db48 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.368404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.369291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.369582 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.369852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.370665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.372567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.399294 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:07.401832 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a147daff9846e699c26259fac4e004'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.409133 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a147daff9846e699c26259fac4e004'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.409981 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a147daff9846e699c26259fac4e004'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.422277 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a147daff9846e699c26259fac4e004'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.461974 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1334/2643] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:07 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:07.463719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.465320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:07.484878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a147daff9846e699c26259fac4e004'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.492265 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1310/2644] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:07 2022] DELETE /identity/v3/users/a0a147daff9846e699c26259fac4e004 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:07.494211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.494426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:07.506610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:07.567252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54d5dc-a28d-4fab-9a0b-472f2316f71f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.586467 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac623e-9332-43e3-be63-e2b1c0e60e88 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.593082 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.593767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.594055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.594337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.595064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.596866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.610931 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1311/2645] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:07.614829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.632143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1f4a7-b994-4e96-a14c-6e4aee6cd8c2 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.642172 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.644112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.645100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.645614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] PATH_INFO: `/v3/projects/c6049d831087477183b8bf4e4660d9a1/users/02b2449bae264725a0730af2ac88c520/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.658567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.659447 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.659711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.662663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.662799 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:07.669164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6049d831087477183b8bf4e4660d9a1, user_id=02b2449bae264725a0730af2ac88c520, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.672348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.677270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.679939 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.693716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1335/2646] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:07.694488 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.695916 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:07.718902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aa3a7-6844-48e6-b9d9-087f893ba126 service placement] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.728817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.729967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.730251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.730536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] PATH_INFO: `/v3/users/43b1ece3cb2e4fd6aa2d711bdff92dce` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.732057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b1ece3cb2e4fd6aa2d711bdff92dce)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.735743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.800408 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1312/2647] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:07 2022] PUT /identity/v3/projects/c6049d831087477183b8bf4e4660d9a1/users/02b2449bae264725a0730af2ac88c520/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:07.801413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:07.804000 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:07.868522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a33c3-b2de-4191-8fff-cac9ff0e378e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:07.873843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:07.874712 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:07.874998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:07.875280 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] PATH_INFO: `/v3/projects/89e9dc06c3304c42a7fb6cbed1e357a4` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:07.886843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e9dc06c3304c42a7fb6cbed1e357a4)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:07.888792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:07.940680 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b1ece3cb2e4fd6aa2d711bdff92dce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.941446 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b1ece3cb2e4fd6aa2d711bdff92dce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.953948 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b1ece3cb2e4fd6aa2d711bdff92dce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:07.960725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b1ece3cb2e4fd6aa2d711bdff92dce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.017096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e9dc06c3304c42a7fb6cbed1e357a4', 'request_id': 'req-db5dceb6-34d8-413e-8258-03864681768e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.022744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b1ece3cb2e4fd6aa2d711bdff92dce'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.029296 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1336/2648] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:07 2022] DELETE /identity/v3/users/43b1ece3cb2e4fd6aa2d711bdff92dce => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:08.030091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.031484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.076390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:08.098844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79b99a-a976-4b14-85eb-6e8ea69d673a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.101526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e9dc06c3304c42a7fb6cbed1e357a4', 'request_id': 'req-db5dceb6-34d8-413e-8258-03864681768e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.103778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.104470 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.104878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.105008 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.105736 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.107844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.110776 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e9dc06c3304c42a7fb6cbed1e357a4', 'request_id': 'req-db5dceb6-34d8-413e-8258-03864681768e'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.118685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1337/2649] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:08.126905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.131899 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:08.135335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] The token cache is being invalidate because project 89e9dc06c3304c42a7fb6cbed1e357a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:08.135651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.136709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-db5dceb6-34d8-413e-8258-03864681768e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.138486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1313/2650] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:07 2022] DELETE /identity/v3/projects/89e9dc06c3304c42a7fb6cbed1e357a4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:08.140047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bc0f0-a2b8-4704-bf5f-2e969c219c71 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.143790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7c01a-eaa6-4946-a627-252d61d1988c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.144145 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7c01a-eaa6-4946-a627-252d61d1988c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.144353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7c01a-eaa6-4946-a627-252d61d1988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.144809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012216fb-5173-4f34-99df-b138f89a800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.145921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012216fb-5173-4f34-99df-b138f89a800c admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.146759 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012216fb-5173-4f34-99df-b138f89a800c admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.147109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012216fb-5173-4f34-99df-b138f89a800c admin admin] PATH_INFO: `/v3/projects/c6049d831087477183b8bf4e4660d9a1/users/02b2449bae264725a0730af2ac88c520/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.149791 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012216fb-5173-4f34-99df-b138f89a800c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6049d831087477183b8bf4e4660d9a1, user_id=02b2449bae264725a0730af2ac88c520, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.153022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012216fb-5173-4f34-99df-b138f89a800c admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.164371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f8c7c01a-eaa6-4946-a627-252d61d1988c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:08.166580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f8c7c01a-eaa6-4946-a627-252d61d1988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.171171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1338/2651] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:08 2022] PUT /identity/v3/projects/c6049d831087477183b8bf4e4660d9a1/users/02b2449bae264725a0730af2ac88c520/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:08.181203 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.182048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.182333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.211694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] MFA Rules not processed for user `02b2449bae264725a0730af2ac88c520`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:08.213006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.241799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1314/2652] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:08.250184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.250486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.250763 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.269141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:08.270398 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.279054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1315/2653] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:39:08.282912 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.287256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1339/2654] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:39:08.293138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4860f-5c5a-4590-8a4b-2d2c942b5f04 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.297483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.298147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.298433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.298716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.299533 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:39:08.299808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.301877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.309912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.309912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.315758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1316/2655] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:39:08.317140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.319885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.334586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516a334-0151-420c-a846-b55103dae17d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.341127 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.341862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.342178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.342460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] PATH_INFO: `/v3/projects/cd5966396128491d814472e980cfd873` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.343729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5966396128491d814472e980cfd873)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.345662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.378195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e234651-e8cd-4ff4-963a-c160191f7506 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.382555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.383210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.383528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.383808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.393959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.398270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.406194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1340/2656] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:39:08.406991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.420351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c57e7-51f2-4da2-85c3-4afe7bab8f33 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.427910 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.432818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.432818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.432818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.432818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.432818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.486730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5966396128491d814472e980cfd873', 'request_id': 'req-32bba6a4-f53d-4e04-af89-da98242e1195'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.534211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1341/2657] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:08 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:08.538375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.548656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5966396128491d814472e980cfd873', 'request_id': 'req-32bba6a4-f53d-4e04-af89-da98242e1195'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.561677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5966396128491d814472e980cfd873', 'request_id': 'req-32bba6a4-f53d-4e04-af89-da98242e1195'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.578338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:08.578566 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] The token cache is being invalidate because project cd5966396128491d814472e980cfd873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:08.578960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.581103 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.585134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1317/2658] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:08 2022] DELETE /identity/v3/projects/cd5966396128491d814472e980cfd873 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:08.588874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.590521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.615813 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:08.632837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b2362-f4dd-4e85-b7e5-df7bf81fc311 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.639900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.645473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.645473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.645473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.645473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.645473 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.674809 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba6a4-f53d-4e04-af89-da98242e1195 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.678368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.679282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.679597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.679873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] PATH_INFO: `/v3/users/c83e7f77ec3c41ff8478f0b955a4c3e7` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.681093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83e7f77ec3c41ff8478f0b955a4c3e7)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.683367 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.763171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1342/2659] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:08 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:08.772100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.774078 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:08.787374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2fb86-a284-41c3-8930-db8e377515a6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.797031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.797902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.798449 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.798917 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.800200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.802278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.818889 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1343/2660] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:39:08.820566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.831507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748b6ca-3e0a-49ea-b174-d478d651375d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.837368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.838514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.838804 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.839083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.839928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.844062 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.857876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1344/2661] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:39:08.862019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:08.873015 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304654-0361-4365-9cd3-fc5a9d373ff4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:08.877871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:08.878816 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:08.879112 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:08.879416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] PATH_INFO: `/v3/projects/756cb4ed8d2345dcaae27638b257a480/users/a17f39a1626d45299666f43be4ffc3ec/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:08.895493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=756cb4ed8d2345dcaae27638b257a480, user_id=a17f39a1626d45299666f43be4ffc3ec, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:08.898205 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:08.939829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83e7f77ec3c41ff8478f0b955a4c3e7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.947718 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83e7f77ec3c41ff8478f0b955a4c3e7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.948370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83e7f77ec3c41ff8478f0b955a4c3e7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:08.961235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83e7f77ec3c41ff8478f0b955a4c3e7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.004623 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1345/2662] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:08 2022] PUT /identity/v3/projects/756cb4ed8d2345dcaae27638b257a480/users/a17f39a1626d45299666f43be4ffc3ec/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:09.005667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:09.007285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.050810 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-26a6a97c-b049-4cab-bf24-5499aefc4b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83e7f77ec3c41ff8478f0b955a4c3e7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.063754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1318/2663] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:08 2022] DELETE /identity/v3/users/c83e7f77ec3c41ff8478f0b955a4c3e7 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:09.063754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.063754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.063754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.082834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:09.084139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.085925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af04f73-3590-4d5d-8f4a-06b57a453e72 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:09.159014 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:09.159776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.163074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.163074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] PATH_INFO: `/v3/users/bcf9566afb2a4e4ea04f6a4705ada3f9` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.163354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1319/2664] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:09.163462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf9566afb2a4e4ea04f6a4705ada3f9)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:09.164118 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:09.169788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:09.171229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.189793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:09.220950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:09.236695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c1e0a-74ce-4f8c-87fd-71c5685921aa None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:09.240438 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:09.241073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.241354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.241629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.251756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:09.253231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:09.264177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1320/2665] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:39:09.339357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:09.344068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.366185 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf9566afb2a4e4ea04f6a4705ada3f9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.367672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf9566afb2a4e4ea04f6a4705ada3f9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.373843 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9395e-4b4b-494c-aaab-5c98e4a22718 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:09.382393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:09.384101 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.385567 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.386078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] PATH_INFO: `/v3/projects/98d44b4a07444128a45ad8eaed75bfa4` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.386880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf9566afb2a4e4ea04f6a4705ada3f9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.388609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d44b4a07444128a45ad8eaed75bfa4)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:09.392867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:09.395360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf9566afb2a4e4ea04f6a4705ada3f9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.463467 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-508f6fdb-981e-4cd0-87d8-3f86f75097fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf9566afb2a4e4ea04f6a4705ada3f9'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.473792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1346/2666] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:09 2022] DELETE /identity/v3/users/bcf9566afb2a4e4ea04f6a4705ada3f9 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:09.478348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74157182-6289-4612-886b-e34bcab60a5b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.478607 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74157182-6289-4612-886b-e34bcab60a5b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.478885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74157182-6289-4612-886b-e34bcab60a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.499953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-74157182-6289-4612-886b-e34bcab60a5b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:09.501168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-74157182-6289-4612-886b-e34bcab60a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.563660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1347/2667] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:39:09.574128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74157182-6289-4612-886b-e34bcab60a5b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:09.575642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-74157182-6289-4612-886b-e34bcab60a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.577117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d44b4a07444128a45ad8eaed75bfa4', 'request_id': 'req-c4a5fc7f-d066-44da-bde1-520b72342eba'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.652332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74157182-6289-4612-886b-e34bcab60a5b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:09.691739 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d44b4a07444128a45ad8eaed75bfa4', 'request_id': 'req-c4a5fc7f-d066-44da-bde1-520b72342eba'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.698312 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d44b4a07444128a45ad8eaed75bfa4', 'request_id': 'req-c4a5fc7f-d066-44da-bde1-520b72342eba'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.707132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a329ed-823a-473a-80b0-9b7ead7c5510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:09.707858 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a329ed-823a-473a-80b0-9b7ead7c5510 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.708088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a329ed-823a-473a-80b0-9b7ead7c5510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.708370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a329ed-823a-473a-80b0-9b7ead7c5510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.709814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] The token cache is being invalidate because project 98d44b4a07444128a45ad8eaed75bfa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:09.710167 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.710867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c4a5fc7f-d066-44da-bde1-520b72342eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:09.716282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1321/2668] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:09 2022] DELETE /identity/v3/projects/98d44b4a07444128a45ad8eaed75bfa4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:09.717855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:09.722306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.722306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.722306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.738219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:09.739462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.778502 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a329ed-823a-473a-80b0-9b7ead7c5510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:09.780066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a329ed-823a-473a-80b0-9b7ead7c5510 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:09.790530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1348/2669] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:39:09.804859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1322/2670] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:39:09.813970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.818284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.818284 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.833125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:09.834325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:09.842530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1349/2671] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:09.846749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:09.857580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd473-22c3-4f94-afd1-54e11180783f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:09.863731 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:09.864708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.864995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.865274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.872755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:39:09.872755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:09.872755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:09.883509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1323/2672] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:39:09.888055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:09.897654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66beab-fc86-4e55-9385-f86a57f40ba5 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:09.903725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:09.904405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:09.904687 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:09.904964 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:09.905714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:09.907777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:09.990674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1350/2673] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:09 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:09.995328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.005812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72af63d-79d8-45db-964c-d598876eb39e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.060501 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:10.071600 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.072443 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.072723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.072997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.073754 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.076024 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:10.153889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1324/2674] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:09 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:10.154545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.167170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdb597-f6d1-41bf-ba99-08d9b05b3979 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.172780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.173594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.173950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.174313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.175106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.176852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:10.192452 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1351/2675] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:39:10.193723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.205008 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d19523-98c3-4d41-8fc3-21c69ebb837b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.209908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385bf04-5741-42bb-ba30-9fd7b04a78c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.210681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385bf04-5741-42bb-ba30-9fd7b04a78c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.211044 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385bf04-5741-42bb-ba30-9fd7b04a78c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.211506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385bf04-5741-42bb-ba30-9fd7b04a78c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.212324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385bf04-5741-42bb-ba30-9fd7b04a78c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.214576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385bf04-5741-42bb-ba30-9fd7b04a78c0 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:10.226299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1325/2676] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:10.231035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.242216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b7513-40e2-4cf8-b7ce-f6f62cdd1710 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.247051 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.247976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.248269 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.248569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] PATH_INFO: `/v3/projects/6d70e82d96624016bed251b30b26ad73/users/44b4ed73a96143098d2c64384233d539/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.265335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d70e82d96624016bed251b30b26ad73, user_id=44b4ed73a96143098d2c64384233d539, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.268047 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:10.355364 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1352/2677] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:10 2022] PUT /identity/v3/projects/6d70e82d96624016bed251b30b26ad73/users/44b4ed73a96143098d2c64384233d539/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:10.361474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.361769 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.362047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.380821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:10.382070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:10.446755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1326/2678] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:10.456868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.458551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:10.518159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec1f26-06e1-4b9b-b1a5-fef091c82b36 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.521921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.522570 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.522855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.523135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.532674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.534110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:10.543933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1353/2679] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:10.553143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.555711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:10.577696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adaa84-ec11-4da8-8a61-b4a6a437072c None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.582451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.583498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.583788 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.584068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] PATH_INFO: `/v3/projects/f0430d8dc58d48f5a74831c36438a98e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.585263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0430d8dc58d48f5a74831c36438a98e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.587295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:10.722509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0430d8dc58d48f5a74831c36438a98e', 'request_id': 'req-e824205a-cf1b-424e-a3ba-b281c754be9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:10.795890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0430d8dc58d48f5a74831c36438a98e', 'request_id': 'req-e824205a-cf1b-424e-a3ba-b281c754be9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:10.804689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0430d8dc58d48f5a74831c36438a98e', 'request_id': 'req-e824205a-cf1b-424e-a3ba-b281c754be9d'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:10.828584 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] The token cache is being invalidate because project f0430d8dc58d48f5a74831c36438a98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:10.828997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:10.829879 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:10.831782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1327/2680] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:10 2022] DELETE /identity/v3/projects/f0430d8dc58d48f5a74831c36438a98e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:10.835813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:10.850437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404372c-0ec8-4763-8449-95cfeb9a0e09 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:10.914282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:10.932993 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:10.933968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:10.934338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:10.934622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] PATH_INFO: `/v3/users/02b2449bae264725a0730af2ac88c520` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:10.935864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b2449bae264725a0730af2ac88c520)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:10.937843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:11.169060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b2449bae264725a0730af2ac88c520'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:11.170928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b2449bae264725a0730af2ac88c520'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:11.187665 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b2449bae264725a0730af2ac88c520'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:11.196152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b2449bae264725a0730af2ac88c520'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:11.274110 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-48fc1e74-7861-44e2-ab35-8ffa0313c147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b2449bae264725a0730af2ac88c520'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:11.282397 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1354/2681] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:10 2022] DELETE /identity/v3/users/02b2449bae264725a0730af2ac88c520 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:11.917565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:11.919466 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:11.934096 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824205a-cf1b-424e-a3ba-b281c754be9d admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:12.000947 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:12.007867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:12.008807 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:12.009092 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:12.010104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] PATH_INFO: `/v3/projects/c6049d831087477183b8bf4e4660d9a1` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:12.011624 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6049d831087477183b8bf4e4660d9a1)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:12.013542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:12.235050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6049d831087477183b8bf4e4660d9a1', 'request_id': 'req-0e48e94b-4f54-4af1-828f-d296b4c3c884'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:12.358986 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6049d831087477183b8bf4e4660d9a1', 'request_id': 'req-0e48e94b-4f54-4af1-828f-d296b4c3c884'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:12.375426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6049d831087477183b8bf4e4660d9a1', 'request_id': 'req-0e48e94b-4f54-4af1-828f-d296b4c3c884'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:12.396117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] The token cache is being invalidate because project c6049d831087477183b8bf4e4660d9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:12.396804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:12.397820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-0e48e94b-4f54-4af1-828f-d296b4c3c884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:12.401317 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1328/2682] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:11 2022] DELETE /identity/v3/projects/c6049d831087477183b8bf4e4660d9a1 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:12.407739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:12.409074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:12.411004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:12.439120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:12.440656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:12.529311 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1355/2683] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:12.537660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:12.543154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:12.543154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:12.565656 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:12.566990 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:12.579416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1329/2684] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:12.582781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:12.593171 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3518e-add0-4252-be70-16efe0046b35 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:12.598307 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:12.599016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:12.599310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:12.599605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:12.600343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:39:12.600652 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:12.602438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:12.622177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:12.633318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1356/2685] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:39:12.653326 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7e3ff-1eea-4c04-afa4-654b09d15376 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:12.659923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:12.660603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:12.660897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:12.661338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:12.662055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:12.664848 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:12.801048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:12.813043 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1330/2686] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:12 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:39:12.819341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a06a-9a75-48c7-bcbc-6384da0af539 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:12.883531 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:12.896268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:12.897083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:12.897371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:12.897648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:12.898344 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:12.900507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:13.062919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1357/2687] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:12 2022] POST /identity/v3/users => generated 288 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:13.067334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:13.081285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb87389-5762-48d1-a0b9-b39e024645de admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:13.107805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:13.108492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:13.108778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:13.109059 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:13.109804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:13.111456 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:13.132373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1331/2688] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:13.137113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:13.160023 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c807ebd-fa96-41d5-b6b5-e34350efb157 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:13.179892 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3186d99-6084-4a98-9136-70a5f7f5de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:13.180595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3186d99-6084-4a98-9136-70a5f7f5de9f admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:13.183088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3186d99-6084-4a98-9136-70a5f7f5de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:13.183450 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3186d99-6084-4a98-9136-70a5f7f5de9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:13.184211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3186d99-6084-4a98-9136-70a5f7f5de9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:13.186120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3186d99-6084-4a98-9136-70a5f7f5de9f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:13.231089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:13.232757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1358/2689] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:13.250147 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf47207-82af-43e1-b5bc-1e4a41c1c801 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:13.258615 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:13.261236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:13.261768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:13.262324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] PATH_INFO: `/v3/projects/779b90003a1c4d9d9a1b05c4dd8f00f2/users/261db5af625b44169e86cca8f32539c7/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:13.288306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=779b90003a1c4d9d9a1b05c4dd8f00f2, user_id=261db5af625b44169e86cca8f32539c7, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:13.293358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:13.396325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1332/2690] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:13 2022] PUT /identity/v3/projects/779b90003a1c4d9d9a1b05c4dd8f00f2/users/261db5af625b44169e86cca8f32539c7/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:13.411370 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:13.411931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:13.411931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:13.443121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] MFA Rules not processed for user `261db5af625b44169e86cca8f32539c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:13.444487 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:13.558665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:13.561187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1359/2691] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:13.707357 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffeaa0-872e-408f-b2ac-f4cd6f033212 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:13.746229 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:13.750209 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:13.751612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:13.751900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:13.754572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:13.761166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:13.866135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1333/2692] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:13 2022] POST /identity/v3/projects => generated 373 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:39:13.867326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:13.891907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af4f16-7ef8-4ca2-ac74-6c94e20b961a None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:14.018671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:14.019516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.019814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.029448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.029448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:14.029448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:14.102520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1360/2693] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:13 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:14.106250 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:14.119609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b382c-af9f-436a-8727-9c9354e71031 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:14.125458 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:14.126598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.127175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.127726 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.128711 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:14.130648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:14.132436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:14.150325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:14.158151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1334/2694] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:14.169318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213794c3-483b-494d-a4c7-f1bb790c5e1d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:14.180128 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:14.181041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.181905 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.182312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] PATH_INFO: `/v3/projects/8397b9876ff04080999ca185116b2023/users/db79a9dd49c3482c915be79d7b7b7e6b/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.226006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8397b9876ff04080999ca185116b2023, user_id=db79a9dd49c3482c915be79d7b7b7e6b, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:14.229489 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:14.311349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1361/2695] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:14 2022] PUT /identity/v3/projects/8397b9876ff04080999ca185116b2023/users/db79a9dd49c3482c915be79d7b7b7e6b/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:14.313055 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:14.323840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f34f24-a69b-4b2f-868d-f9925a8a235c admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:14.398834 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c81219-be6d-4394-855f-5de3c96ea3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:14.399847 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c81219-be6d-4394-855f-5de3c96ea3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.400171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c81219-be6d-4394-855f-5de3c96ea3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.400506 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c81219-be6d-4394-855f-5de3c96ea3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.401463 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c81219-be6d-4394-855f-5de3c96ea3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:14.403154 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c81219-be6d-4394-855f-5de3c96ea3b8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:14.422428 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1335/2696] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:14.424189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:14.447996 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05909a2-bbd2-4f62-a5ca-afab195f5fa9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:14.456605 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:14.458365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.458757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.459167 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] PATH_INFO: `/v3/projects/8397b9876ff04080999ca185116b2023/users/db79a9dd49c3482c915be79d7b7b7e6b/roles/02d9e6449ea143beb203824112640698` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.462067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8397b9876ff04080999ca185116b2023, user_id=db79a9dd49c3482c915be79d7b7b7e6b, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:14.467616 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:14.482483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1362/2697] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:14 2022] PUT /identity/v3/projects/8397b9876ff04080999ca185116b2023/users/db79a9dd49c3482c915be79d7b7b7e6b/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:14.492859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9192c-80f3-4322-b9c2-f15a542ccde7 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.493211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9192c-80f3-4322-b9c2-f15a542ccde7 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.493504 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9192c-80f3-4322-b9c2-f15a542ccde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.540151 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-53e9192c-80f3-4322-b9c2-f15a542ccde7 None None] MFA Rules not processed for user `db79a9dd49c3482c915be79d7b7b7e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:14.542351 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-53e9192c-80f3-4322-b9c2-f15a542ccde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:14.635575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1336/2698] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:14.647486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:14.649388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:14.692091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:14.804347 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcc814-a885-410d-8a5a-dce813ec849d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:14.808357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:14.809031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:14.809336 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:14.809967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:14.943571 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:14.945065 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:14.958535 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1363/2699] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:16.550328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff15688-4097-40b9-8dbf-181e3e7b9a03 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:16.553224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff15688-4097-40b9-8dbf-181e3e7b9a03 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:16.553224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff15688-4097-40b9-8dbf-181e3e7b9a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:16.584709 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-eff15688-4097-40b9-8dbf-181e3e7b9a03 None None] MFA Rules not processed for user `a17f39a1626d45299666f43be4ffc3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:16.585993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-eff15688-4097-40b9-8dbf-181e3e7b9a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:16.669426 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1337/2700] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:16.689024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:16.691443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:16.771728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a7b70-788a-4a65-b9ef-9e7ff46c6494 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:16.779609 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:16.780281 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:16.780566 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:16.780840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:16.791519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:16.793260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:16.803694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1364/2701] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:18.137699 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:18.138410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:18.138410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:18.166123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:18.191732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] MFA Rules not processed for user `44b4ed73a96143098d2c64384233d539`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:18.193374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:18.327762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1338/2702] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:18.332056 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:18.333704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:18.437655 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:18.471968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ec7d9-cd2a-4038-8c49-0400535fd2a6 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:18.498401 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:18.499145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:18.499498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:18.500548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:18.500548 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:18.509260 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:18.653525 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1365/2703] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:18 2022] POST /identity/v3/projects => generated 371 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:18.655226 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:18.674673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b470cd-4e60-4982-9ef3-5264a1bb53cc None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:18.802361 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:18.803161 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:18.803603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:18.803977 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:18.804733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:18.806742 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:18.947564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1339/2704] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:18 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:18.952184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:18.965116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7bbc-9eea-414d-80df-e96e367defd1 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:18.971176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:18.972741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:18.972741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:18.972741 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:18.973388 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:18.975403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:19.001024 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1366/2705] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:39:19.006354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:19.022098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae692e-2c14-4865-9836-89ac8d9a64c7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:19.032840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:19.033852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:19.038301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:19.038301 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] PATH_INFO: `/v3/projects/e02b170043e14f41923636805f4e2a15/users/b644b980149f4ae4a1b469ea78a8718a/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:19.053636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02b170043e14f41923636805f4e2a15, user_id=b644b980149f4ae4a1b469ea78a8718a, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:19.057100 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:19.113676 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1340/2706] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:19 2022] PUT /identity/v3/projects/e02b170043e14f41923636805f4e2a15/users/b644b980149f4ae4a1b469ea78a8718a/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:19.118122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:19.138549 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed792b0-6f1d-45c5-8f1a-f24101b98bd8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:19.285563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:19.290742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:19.290742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:19.290742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:19.290742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:19.294972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:19.327543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1367/2707] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:19.330933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:19.351335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1970f-bff2-4a1d-a191-0f646b0506de admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:19.362729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e73aa6-5aac-4836-863b-b663279180a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:19.363751 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e73aa6-5aac-4836-863b-b663279180a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:19.364034 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e73aa6-5aac-4836-863b-b663279180a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:19.365440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e73aa6-5aac-4836-863b-b663279180a2 admin admin] PATH_INFO: `/v3/projects/e02b170043e14f41923636805f4e2a15/users/b644b980149f4ae4a1b469ea78a8718a/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:19.367556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e73aa6-5aac-4836-863b-b663279180a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02b170043e14f41923636805f4e2a15, user_id=b644b980149f4ae4a1b469ea78a8718a, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:19.371051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e73aa6-5aac-4836-863b-b663279180a2 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:19.387845 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1341/2708] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:19 2022] PUT /identity/v3/projects/e02b170043e14f41923636805f4e2a15/users/b644b980149f4ae4a1b469ea78a8718a/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:19.951867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:19.968252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10232bf1-5f5a-4b7b-9ac6-8291afd784d8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:20.115144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:20.117454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:20.118071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:20.118716 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] PATH_INFO: `/v3/users/261db5af625b44169e86cca8f32539c7` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:20.120808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=261db5af625b44169e86cca8f32539c7)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:20.125004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:20.240716 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae3651-d946-4083-9e6f-0bfa65d260c0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:20.241093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae3651-d946-4083-9e6f-0bfa65d260c0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:20.241558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae3651-d946-4083-9e6f-0bfa65d260c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:20.272978 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-d6ae3651-d946-4083-9e6f-0bfa65d260c0 None None] MFA Rules not processed for user `a17f39a1626d45299666f43be4ffc3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:20.274273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d6ae3651-d946-4083-9e6f-0bfa65d260c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:20.360278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:20.375505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261db5af625b44169e86cca8f32539c7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:20.377000 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261db5af625b44169e86cca8f32539c7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:20.392766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1342/2709] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:20.403778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:20.406394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261db5af625b44169e86cca8f32539c7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:20.422358 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261db5af625b44169e86cca8f32539c7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:20.496383 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261db5af625b44169e86cca8f32539c7'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:20.505122 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1368/2710] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:19 2022] DELETE /identity/v3/users/261db5af625b44169e86cca8f32539c7 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:20.508273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:20.508816 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:20.509851 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:20.534883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:20.536377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:20.604480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1343/2711] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:20.614618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:20.616201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:20.694627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be64c-328e-40ef-a5fc-a06585955846 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:20.700108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:20.701244 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:20.701792 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:20.702289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:20.713721 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:20.715183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:20.729500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1369/2712] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:21.215017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:21.217527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:21.264882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:21.267589 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:21.299465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e915362-0891-4b1a-a633-5048b47ef2d1 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:21.307972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:21.309338 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:21.309781 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:21.312145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] PATH_INFO: `/v3/projects/779b90003a1c4d9d9a1b05c4dd8f00f2` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:21.312145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=779b90003a1c4d9d9a1b05c4dd8f00f2)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:21.314697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:21.340246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38679070-fb3d-4a9a-b233-86da46bf607f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:21.354826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:21.355831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:21.357098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:21.357604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:21.359969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:21.375802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:21.378173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:21.392370 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1344/2713] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:21.501597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '779b90003a1c4d9d9a1b05c4dd8f00f2', 'request_id': 'req-7b25f174-1675-4535-ac2d-f5d3c80791b6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:21.517320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '779b90003a1c4d9d9a1b05c4dd8f00f2', 'request_id': 'req-7b25f174-1675-4535-ac2d-f5d3c80791b6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:21.600586 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '779b90003a1c4d9d9a1b05c4dd8f00f2', 'request_id': 'req-7b25f174-1675-4535-ac2d-f5d3c80791b6'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:21.634614 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] The token cache is being invalidate because project 779b90003a1c4d9d9a1b05c4dd8f00f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:21.635039 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:21.642507 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:21.647235 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1370/2714] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:21 2022] DELETE /identity/v3/projects/779b90003a1c4d9d9a1b05c4dd8f00f2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:21.648629 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:21.662121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b49e5-3cb1-4b5c-85e1-09c6efab81e2 service placement] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:21.746350 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:21.747256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:21.747577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:21.747908 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] PATH_INFO: `/v3/users/db79a9dd49c3482c915be79d7b7b7e6b` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:21.752486 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db79a9dd49c3482c915be79d7b7b7e6b)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:21.754760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:21.908471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:22.108341 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db79a9dd49c3482c915be79d7b7b7e6b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:22.133002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db79a9dd49c3482c915be79d7b7b7e6b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:22.134025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db79a9dd49c3482c915be79d7b7b7e6b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:22.154721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db79a9dd49c3482c915be79d7b7b7e6b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:22.234755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-84546cb1-5121-40a3-9098-bfe33d0e1221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db79a9dd49c3482c915be79d7b7b7e6b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:22.242149 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1345/2715] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:21 2022] DELETE /identity/v3/users/db79a9dd49c3482c915be79d7b7b7e6b => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:22.821929 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:22.824679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:22.858479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f174-1675-4535-ac2d-f5d3c80791b6 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:22.973895 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:22.974795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:22.975127 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:22.975528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] PATH_INFO: `/v3/projects/8397b9876ff04080999ca185116b2023` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:22.976754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8397b9876ff04080999ca185116b2023)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:22.979840 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:22.987376 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:23.168465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8397b9876ff04080999ca185116b2023', 'request_id': 'req-9005a564-ec02-44f2-b680-83ccdacb9250'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:23.189001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8397b9876ff04080999ca185116b2023', 'request_id': 'req-9005a564-ec02-44f2-b680-83ccdacb9250'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:23.255519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8397b9876ff04080999ca185116b2023', 'request_id': 'req-9005a564-ec02-44f2-b680-83ccdacb9250'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:23.272747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] The token cache is being invalidate because project 8397b9876ff04080999ca185116b2023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:23.273323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:23.273999 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-9005a564-ec02-44f2-b680-83ccdacb9250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:23.276192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1371/2716] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:22 2022] DELETE /identity/v3/projects/8397b9876ff04080999ca185116b2023 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:23.281163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:23.282097 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:23.282382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:23.302717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:23.303964 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:23.372480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1346/2717] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:23.380457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:23.381206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:23.381206 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:23.410429 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:23.412247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:23.445211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:23.452088 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1372/2718] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:39:23.457956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96029973-bc94-4571-ae4a-fb3b39005f30 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:23.464246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:23.465633 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:23.466014 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:23.466397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:23.467354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:39:23.467878 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:23.470715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:23.489846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1347/2719] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 01 04:39:23.496031 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:23.521318 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348acc32-c0e8-4530-8994-f5916419adfe None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:23.528628 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:23.529762 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:23.530240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:23.530668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:23.531728 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:23.534268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:23.637613 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1373/2720] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:23 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:23.641765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:23.660909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf727b-8189-4927-b4bf-5d0d863a3932 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:23.776354 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:23.777224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:23.779122 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:23.779681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:23.780427 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:23.783193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:23.949314 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1348/2721] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:23 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:23.952980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:24.008734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b93c7b-f013-480a-bb62-bfd5bbd6412f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:24.015697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:24.016647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:24.017093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:24.017536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:24.019214 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:24.021594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:24.037103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1374/2722] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:24.041588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:24.076673 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:24.094180 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cff814-e8f1-46e0-953e-799a60114c00 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:24.099901 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56795870-94b8-4ceb-926a-aa681dd7c615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:24.100607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56795870-94b8-4ceb-926a-aa681dd7c615 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:24.104091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56795870-94b8-4ceb-926a-aa681dd7c615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:24.104091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56795870-94b8-4ceb-926a-aa681dd7c615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:24.104091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56795870-94b8-4ceb-926a-aa681dd7c615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:24.104091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56795870-94b8-4ceb-926a-aa681dd7c615 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:24.120705 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1349/2723] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:24.121541 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:24.139913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892954d-7028-47b0-9154-bc2c37514a0e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:24.147396 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:24.152650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:24.153170 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:24.153530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] PATH_INFO: `/v3/projects/dc63b8a952e54ed59ff19757e3e68f69/users/b35023912ad843b191d9d3eb8a3a5331/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:24.192168 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63b8a952e54ed59ff19757e3e68f69, user_id=b35023912ad843b191d9d3eb8a3a5331, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:24.197928 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:24.310938 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:24.311916 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:24.312433 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:24.316879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1375/2724] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:24 2022] PUT /identity/v3/projects/dc63b8a952e54ed59ff19757e3e68f69/users/b35023912ad843b191d9d3eb8a3a5331/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:24.353896 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] MFA Rules not processed for user `b35023912ad843b191d9d3eb8a3a5331`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:24.359865 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:24.492803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:24.493320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1350/2725] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:24.582546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f70f1-088e-45c5-a2f5-ffe07916a96a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:24.596037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:24.596664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:24.597105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:24.597363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:24.598067 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:24.600029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:24.633017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:24.762142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1376/2726] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:24 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:24.765928 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:24.786578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108e48e-28af-495f-a215-a344dab4b603 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:24.902787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:24.907372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:24.907781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:24.908073 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:24.908794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:24.914219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:25.025568 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1351/2727] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:24 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:25.026810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:25.044406 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b0b94-23e8-4efc-88c0-84b046d16dc3 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:25.060815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:25.061657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:25.061951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:25.062236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:25.063142 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:25.065551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:25.089872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:25.095161 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1377/2728] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:25.103872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d23bd-57c8-4375-9c13-8c0b7ba9d50a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:25.125564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218b326-62a0-4c7e-a16c-926ccd77096f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:25.126587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b326-62a0-4c7e-a16c-926ccd77096f admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:25.126877 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b326-62a0-4c7e-a16c-926ccd77096f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:25.127156 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218b326-62a0-4c7e-a16c-926ccd77096f admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:25.127926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218b326-62a0-4c7e-a16c-926ccd77096f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:25.129678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218b326-62a0-4c7e-a16c-926ccd77096f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:25.154159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1352/2729] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:39:25.158437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:25.174182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08dde4-24ec-4b9a-889f-7a50f37c1437 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:25.187475 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:25.188874 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:25.189320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:25.189778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] PATH_INFO: `/v3/projects/ca248e145d66486485653924682f41cf/users/f21053d2ce5a43f5b5ffddc259c81d9b/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:25.220890 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca248e145d66486485653924682f41cf, user_id=f21053d2ce5a43f5b5ffddc259c81d9b, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:25.223615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:25.289427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1378/2730] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:25 2022] PUT /identity/v3/projects/ca248e145d66486485653924682f41cf/users/f21053d2ce5a43f5b5ffddc259c81d9b/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:25.302171 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:25.302528 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:25.302749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:25.360488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] MFA Rules not processed for user `f21053d2ce5a43f5b5ffddc259c81d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:25.362446 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:25.423648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:25.496311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1353/2731] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:25.500495 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:25.633279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161ccc-9c5d-40c9-a239-1776509c968f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:25.648809 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:25.649506 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:25.650357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:25.650644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:25.656443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:25.692588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:25.813077 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1379/2732] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:25 2022] POST /identity/v3/projects => generated 385 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:25.816987 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:25.834576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8097c-d334-4cbc-a92e-20c76f493ff2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:25.980403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:25.981042 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:25.981593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:25.981873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:25.982733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:25.996675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:26.090592 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1354/2733] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:25 2022] POST /identity/v3/users => generated 299 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:26.094815 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:26.111094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d6f5e-ce0e-4df8-8e53-0f415659e94d admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:26.131936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:26.132705 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:26.133152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:26.133704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:26.135275 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:26.138231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:26.152276 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1380/2734] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:26.156214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:26.174419 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41de8cd-2f86-4f0c-b154-121c43a0990f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:26.180181 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:26.181914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:26.182373 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:26.182800 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] PATH_INFO: `/v3/projects/f35c9d51df554edd88b8109abdd4419d/users/cc71f993bae7479a84191817f51a92c0/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:26.240505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35c9d51df554edd88b8109abdd4419d, user_id=cc71f993bae7479a84191817f51a92c0, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:26.245730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:26.338079 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1355/2735] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:26 2022] PUT /identity/v3/projects/f35c9d51df554edd88b8109abdd4419d/users/cc71f993bae7479a84191817f51a92c0/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:26.341860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:26.356780 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a6cc0-59f5-4715-9e30-961b1ae65bd4 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:26.478729 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e1e4a-590c-4cd2-8126-3144cd4c74b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:26.479457 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e1e4a-590c-4cd2-8126-3144cd4c74b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:26.480776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e1e4a-590c-4cd2-8126-3144cd4c74b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:26.481060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e1e4a-590c-4cd2-8126-3144cd4c74b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:26.482155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e1e4a-590c-4cd2-8126-3144cd4c74b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:26.485649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e1e4a-590c-4cd2-8126-3144cd4c74b9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:26.506869 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1381/2736] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:26.508146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:26.523621 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d66a5c-6c84-4791-9e8a-989ef58f6fe8 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:26.533077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:26.538734 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:26.543359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:26.543359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] PATH_INFO: `/v3/projects/f35c9d51df554edd88b8109abdd4419d/users/cc71f993bae7479a84191817f51a92c0/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:26.546451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35c9d51df554edd88b8109abdd4419d, user_id=cc71f993bae7479a84191817f51a92c0, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:26.556338 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:26.577332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1356/2737] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:26 2022] PUT /identity/v3/projects/f35c9d51df554edd88b8109abdd4419d/users/cc71f993bae7479a84191817f51a92c0/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:26.592346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317e8f0-d0da-4fe1-a4ea-b943c6944add None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:26.592642 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317e8f0-d0da-4fe1-a4ea-b943c6944add None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:26.592957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317e8f0-d0da-4fe1-a4ea-b943c6944add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:26.618294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:26.636184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-8317e8f0-d0da-4fe1-a4ea-b943c6944add None None] MFA Rules not processed for user `cc71f993bae7479a84191817f51a92c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:26.638002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8317e8f0-d0da-4fe1-a4ea-b943c6944add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:26.738803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1382/2738] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:26.754811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:26.756867 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:26.941017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee20059-399a-478f-8170-5450c5af23a7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:26.945682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:26.945682 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:26.947039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:26.947039 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:26.990645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:27.031135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:27.033532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:27.045025 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1357/2739] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:27.246875 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:27.248363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:27.249025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:27.309606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] MFA Rules not processed for user `b644b980149f4ae4a1b469ea78a8718a`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:27.311327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:27.513766 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1383/2740] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:27.526625 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:27.529887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:27.719465 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397f93d-d2fb-4dc5-ab0e-8986e455470c service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:27.728371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:27.729184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:27.729526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:27.729803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:27.737094 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:27.812759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:27.815020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42fac8-2bc2-4c3d-9699-2bac280f8b45 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:27.827531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1358/2741] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:30.463283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:30.473530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c584cf4-d2d7-423c-9c26-2a3a52d2afee None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:30.477411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:30.478109 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:30.478390 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:30.478663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:30.557200 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:30.558668 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:30.568882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1384/2742] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:30.638391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:30.638837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:30.638949 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:30.668660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] MFA Rules not processed for user `44b4ed73a96143098d2c64384233d539`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:30.669576 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:30.677483 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1359/2743] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:32.345526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:32.355934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c88c18-6e6c-4f7d-958b-f7c584ca9877 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:32.360120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:32.360778 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:32.361060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:32.361346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:32.373744 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:32.375208 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:32.386534 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1385/2744] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:33.218386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:33.221439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:33.238487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be498d-aea9-44a9-88a9-a4a2f52f7a18 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:33.245105 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:33.245396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:33.245677 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:33.245952 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:33.247775 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:33.261217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:33.263175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd846e9e-020f-4aee-a9fa-e17984c63adc service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:33.278837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1360/2745] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:38.282867 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:38.286640 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:38.379817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682895e-1d20-4fb2-80e1-78b6cbc54cf0 service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:38.385050 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:38.385925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:38.386202 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:38.386672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] PATH_INFO: `/v3/users/b35023912ad843b191d9d3eb8a3a5331` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:38.388104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35023912ad843b191d9d3eb8a3a5331)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:38.390035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:38.642674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35023912ad843b191d9d3eb8a3a5331'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:38.643615 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35023912ad843b191d9d3eb8a3a5331'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:38.657156 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35023912ad843b191d9d3eb8a3a5331'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:38.664002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35023912ad843b191d9d3eb8a3a5331'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:38.730226 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35023912ad843b191d9d3eb8a3a5331'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:38.736661 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1386/2746] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:38 2022] DELETE /identity/v3/users/b35023912ad843b191d9d3eb8a3a5331 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:38.741556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:38.742062 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:38.742195 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:38.772436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:38.773655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:38.814829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=100 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:38.834028 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1361/2747] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:38.845046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:38.848060 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:38.928501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:38.937464 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc570ef-2436-40d7-8b76-72cb5d754a97 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:38.941274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:38.941925 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:38.942211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:38.942490 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:38.958293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:38.958293 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:38.965503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1387/2748] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:39.427799 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:39.429256 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:39.442932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b0e48-c01f-4ca4-8ddb-9a93eebbc1c5 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:39.448577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:39.450404 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:39.450895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:39.451247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] PATH_INFO: `/v3/projects/dc63b8a952e54ed59ff19757e3e68f69` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:39.452535 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc63b8a952e54ed59ff19757e3e68f69)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:39.454922 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:39.612761 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc63b8a952e54ed59ff19757e3e68f69', 'request_id': 'req-97a87511-d12b-423c-8e0e-8afb91c6e25f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:39.702293 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc63b8a952e54ed59ff19757e3e68f69', 'request_id': 'req-97a87511-d12b-423c-8e0e-8afb91c6e25f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:39.715513 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc63b8a952e54ed59ff19757e3e68f69', 'request_id': 'req-97a87511-d12b-423c-8e0e-8afb91c6e25f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:39.727409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] The token cache is being invalidate because project dc63b8a952e54ed59ff19757e3e68f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:39.727762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:39.728407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:39.730053 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1362/2749] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:39 2022] DELETE /identity/v3/projects/dc63b8a952e54ed59ff19757e3e68f69 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:39:39.734912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:39.749091 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda0eb0-edc6-4b48-b21b-8b591d40875f service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:39.822896 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:39.823713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:39.823998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:39.824279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] PATH_INFO: `/v3/users/f21053d2ce5a43f5b5ffddc259c81d9b` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:39.825472 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21053d2ce5a43f5b5ffddc259c81d9b)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:39.827516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:40.058657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21053d2ce5a43f5b5ffddc259c81d9b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.059320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21053d2ce5a43f5b5ffddc259c81d9b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.072252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21053d2ce5a43f5b5ffddc259c81d9b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.084942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21053d2ce5a43f5b5ffddc259c81d9b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.164527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21053d2ce5a43f5b5ffddc259c81d9b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.171231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1388/2750] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:39 2022] DELETE /identity/v3/users/f21053d2ce5a43f5b5ffddc259c81d9b => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:40.599069 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:40.600616 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:40.614854 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87511-d12b-423c-8e0e-8afb91c6e25f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:40.693975 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:40.696702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:40.696702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:40.696702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] PATH_INFO: `/v3/projects/ca248e145d66486485653924682f41cf` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:40.698583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca248e145d66486485653924682f41cf)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:40.700773 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:40.805805 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca248e145d66486485653924682f41cf', 'request_id': 'req-45160d5d-e719-4797-8674-20e2ad5bfbb7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.896126 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca248e145d66486485653924682f41cf', 'request_id': 'req-45160d5d-e719-4797-8674-20e2ad5bfbb7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.903872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca248e145d66486485653924682f41cf', 'request_id': 'req-45160d5d-e719-4797-8674-20e2ad5bfbb7'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.911416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:40.916597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] The token cache is being invalidate because project ca248e145d66486485653924682f41cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:40.916895 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.917732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:40.919610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1363/2751] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:40 2022] DELETE /identity/v3/projects/ca248e145d66486485653924682f41cf => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:40.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:40.928805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:40.938013 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d06a2-54ee-4cf2-abd3-b2bfb319b438 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:40.999647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:41.000587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:41.001017 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:41.001422 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] PATH_INFO: `/v3/users/cc71f993bae7479a84191817f51a92c0` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:41.002819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc71f993bae7479a84191817f51a92c0)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:41.007341 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:41.013620 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:41.213632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc71f993bae7479a84191817f51a92c0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:41.214369 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc71f993bae7479a84191817f51a92c0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:41.225901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc71f993bae7479a84191817f51a92c0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:41.235394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc71f993bae7479a84191817f51a92c0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:41.313131 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-01c30738-c5a6-4f09-b33a-f8c8bafb114d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc71f993bae7479a84191817f51a92c0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:41.320314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1389/2752] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:39:40 2022] DELETE /identity/v3/users/cc71f993bae7479a84191817f51a92c0 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:41.743652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:41.745235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:41.758413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45160d5d-e719-4797-8674-20e2ad5bfbb7 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:41.828645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:41.829522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:41.829803 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:41.830076 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] PATH_INFO: `/v3/projects/f35c9d51df554edd88b8109abdd4419d` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:41.831274 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35c9d51df554edd88b8109abdd4419d)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:41.835874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:41.965208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35c9d51df554edd88b8109abdd4419d', 'request_id': 'req-ce674b6c-f500-48b8-bd75-959a617c1265'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:42.067664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35c9d51df554edd88b8109abdd4419d', 'request_id': 'req-ce674b6c-f500-48b8-bd75-959a617c1265'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:42.073839 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35c9d51df554edd88b8109abdd4419d', 'request_id': 'req-ce674b6c-f500-48b8-bd75-959a617c1265'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:42.085118 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] The token cache is being invalidate because project f35c9d51df554edd88b8109abdd4419d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:39:42.085407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:42.086041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ce674b6c-f500-48b8-bd75-959a617c1265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:39:42.087701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1364/2753] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:39:41 2022] DELETE /identity/v3/projects/f35c9d51df554edd88b8109abdd4419d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:42.092830 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.093285 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.093384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.134785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:42.136283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:42.198748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1390/2754] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:42.206982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.207273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.207593 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.225554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:42.228234 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:42.235819 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1365/2755] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:42.240216 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.250536 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a46e0-c887-48ae-a44e-a8486172de6b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.255176 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.255873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.256160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.256439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.260803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:39:42.260803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.260803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.270882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1391/2756] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:39:42.275790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.284954 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe5c75-8b82-45e2-96db-f1f046b4dec9 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.289586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.290852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.291136 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.291464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.292173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.294064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.372121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1366/2757] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:39:42 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:42.376330 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.386187 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e0cc-1e68-423d-9ad0-9b41d92e9a27 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.441210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:42.446935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.447648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.447942 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.448220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.449002 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.450927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.552484 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1392/2758] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:39:42 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:39:42.556782 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.567324 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19248ec4-158d-4be9-8d7a-27b0234337ac admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.571866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.572524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.572820 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.573075 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.573790 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.575498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.585841 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1367/2759] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:42.591159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.601667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5cd19a-d682-4de6-b015-0bb27e55649b admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.607194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667551b4-d911-403a-85e3-f88390fd0e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.607880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667551b4-d911-403a-85e3-f88390fd0e44 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.608163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667551b4-d911-403a-85e3-f88390fd0e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.608439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667551b4-d911-403a-85e3-f88390fd0e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.609191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667551b4-d911-403a-85e3-f88390fd0e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.610845 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667551b4-d911-403a-85e3-f88390fd0e44 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.621509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1393/2760] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:39:42.625678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.636320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849f7f0-16ee-4ade-8641-a128dd58cbcd admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.640992 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.641914 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.642224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.642550 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] PATH_INFO: `/v3/projects/6607408c2a46439d93026002d312b9de/users/80b97ad4fa674391a99cf97bc10da8bc/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.657017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6607408c2a46439d93026002d312b9de, user_id=80b97ad4fa674391a99cf97bc10da8bc, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.659689 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.742232 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1368/2761] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:39:42 2022] PUT /identity/v3/projects/6607408c2a46439d93026002d312b9de/users/80b97ad4fa674391a99cf97bc10da8bc/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:39:42.747971 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bba97-bda2-4de2-a316-24f0e586f20e None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.748273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bba97-bda2-4de2-a316-24f0e586f20e None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.748578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bba97-bda2-4de2-a316-24f0e586f20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.766351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-1f6bba97-bda2-4de2-a316-24f0e586f20e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:42.767588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1f6bba97-bda2-4de2-a316-24f0e586f20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:42.832174 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1394/2762] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:42.841792 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:42.843602 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:42.872608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:39:42.901019 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbffa4-ed4b-435e-9587-8fab7866dc81 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:42.905231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:42.905887 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:42.906176 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:42.906462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:42.916052 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:42.917487 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:42.926801 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1369/2763] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:44.166004 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c3fb-d1b8-4dc3-86b6-f956e95c34ff None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:44.166529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c3fb-d1b8-4dc3-86b6-f956e95c34ff None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:44.166529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2c3fb-d1b8-4dc3-86b6-f956e95c34ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:44.201138 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-b7a2c3fb-d1b8-4dc3-86b6-f956e95c34ff None None] MFA Rules not processed for user `80b97ad4fa674391a99cf97bc10da8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:44.202356 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b7a2c3fb-d1b8-4dc3-86b6-f956e95c34ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:44.278162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1395/2764] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:44.288941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:44.291206 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:44.349640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5f2cb-bce7-4260-a094-fbc7f17ed11f service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:44.353920 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:44.354575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:44.354859 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:44.355134 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:44.364333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:44.366063 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:44.376359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1370/2765] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:46.128240 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157bdf9-20e5-47d4-9fb3-9daa07b9c323 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:46.128803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157bdf9-20e5-47d4-9fb3-9daa07b9c323 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:46.129178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157bdf9-20e5-47d4-9fb3-9daa07b9c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:46.148742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a157bdf9-20e5-47d4-9fb3-9daa07b9c323 None None] MFA Rules not processed for user `80b97ad4fa674391a99cf97bc10da8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:39:46.149998 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a157bdf9-20e5-47d4-9fb3-9daa07b9c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:46.157343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1396/2766] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:39:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:39:54.047285 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:39:54.057844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2ddfb-0a80-410c-99ac-c8b5c8e41a45 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:39:54.061439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:39:54.062085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:39:54.062372 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:39:54.062653 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:39:54.064539 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:39:54.075275 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:39:54.076738 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:39:54.084994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1371/2767] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:39:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:39:56.142945 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:04.081374 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=47 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:06.768310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:06.768783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:06.768885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:06.787193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] MFA Rules not processed for user `80b97ad4fa674391a99cf97bc10da8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:40:06.788413 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:06.795591 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1397/2768] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:40:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:40:13.861492 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:13.863599 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:13.950645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d617bc1-7523-452c-a455-da9f37cbb783 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:13.956951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:13.960603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:13.960603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:13.960603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:13.960603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:13.973826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:13.975480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:13.987284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1372/2769] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:40:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:40:16.781985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:18.746765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:18.748198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:18.760283 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d88b-e3e2-411d-9dfa-c6dfd755e4cd None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:18.763954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:18.764629 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:18.764915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:18.765193 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:18.767019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:18.829673 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:18.831800 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc172d-fe6c-4811-8648-a35b1a775743 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:18.840793 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1398/2770] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:40:18 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:40:23.983247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=34 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:28.836415 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=30 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:31.657746 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:31.667360 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de36da-b64d-4f54-a48c-0d461078dd44 service nova] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:31.671188 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:31.671927 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:31.672219 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:31.672502 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:31.674606 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:31.688212 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:31.689645 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:31.699804 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1373/2771] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:40:37.274943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:37.275455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:37.275583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:37.335857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:40:37.337071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:37.408500 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1399/2772] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jun 1 04:40:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:40:37.608777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:37.610778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:37.735714 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb29ad1-1d26-476f-ba4f-fa400850fc21 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:37.740760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:37.741406 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:37.741688 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:37.741962 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:37.750214 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:37.751817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:37.759736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1374/2773] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:40:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:40:38.916345 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:38.919678 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:39.016908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3cba3-9f85-40f1-87c5-e6dd32c6f40c None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:39.022886 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:39.023601 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:39.023921 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:39.024225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:39.026135 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:39.042074 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:39.044756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:39.050479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:39.052005 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:39.071912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1400/2774] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:40:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:40:39.076497 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e71c0-e8fd-44a7-8c71-ea04b629bc0f service swift] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:39.082194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:39.083141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:39.083489 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:39.083777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] PATH_INFO: `/v3/users/c052b2f518064843928b4eff569e913f` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:39.104104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c052b2f518064843928b4eff569e913f)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:39.107356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:39.327860 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c052b2f518064843928b4eff569e913f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:39.337166 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c052b2f518064843928b4eff569e913f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:39.337937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c052b2f518064843928b4eff569e913f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:39.352696 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c052b2f518064843928b4eff569e913f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:39.432838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c052b2f518064843928b4eff569e913f'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:39.443583 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1375/2775] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:40:39 2022] DELETE /identity/v3/users/c052b2f518064843928b4eff569e913f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:40:39.793906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:39.810367 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d4be3-54c9-4db7-bafb-71979fc2f4c5 service placement] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:39.894722 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=100 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:39.908739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:39.909624 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:39.909915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:39.910247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] PATH_INFO: `/v3/projects/9369d530285849b3a9eac5e0034d3170` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:39.922343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9369d530285849b3a9eac5e0034d3170)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:39.924770 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:40.086106 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9369d530285849b3a9eac5e0034d3170', 'request_id': 'req-581e7e46-239f-4e46-98aa-8d9495657c96'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.093516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9369d530285849b3a9eac5e0034d3170', 'request_id': 'req-581e7e46-239f-4e46-98aa-8d9495657c96'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.190919 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9369d530285849b3a9eac5e0034d3170', 'request_id': 'req-581e7e46-239f-4e46-98aa-8d9495657c96'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.207952 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] The token cache is being invalidate because project 9369d530285849b3a9eac5e0034d3170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:40:40.208521 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.209266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.212097 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1401/2776] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:40:39 2022] DELETE /identity/v3/projects/9369d530285849b3a9eac5e0034d3170 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:40:40.216978 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:40.219057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:40.241093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960d988-1617-4d8f-83a4-f5e36ac86197 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:40.269886 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:40.347380 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:40.351020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:40.351728 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:40.355124 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] PATH_INFO: `/v3/users/f7289e7b11b6428b8295a05748ebd882` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:40.369471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7289e7b11b6428b8295a05748ebd882)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:40.371640 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:40.636781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7289e7b11b6428b8295a05748ebd882'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.645736 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7289e7b11b6428b8295a05748ebd882'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.647123 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7289e7b11b6428b8295a05748ebd882'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.663678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7289e7b11b6428b8295a05748ebd882'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.946231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-c2c99602-3a73-4be4-a09c-c6abe7597b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7289e7b11b6428b8295a05748ebd882'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:40.953815 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1376/2777] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:40:40 2022] DELETE /identity/v3/users/f7289e7b11b6428b8295a05748ebd882 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:40:41.223551 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:41.226813 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:41.244368 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e7e46-239f-4e46-98aa-8d9495657c96 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:41.315519 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:41.316403 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:41.316698 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:41.316976 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] PATH_INFO: `/v3/projects/f6280619df0e4dac84868ea94e7885e8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:41.335654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6280619df0e4dac84868ea94e7885e8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:41.337841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:41.456052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6280619df0e4dac84868ea94e7885e8', 'request_id': 'req-843286f5-f23b-45c1-a74d-47fd0378bb59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:41.466144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6280619df0e4dac84868ea94e7885e8', 'request_id': 'req-843286f5-f23b-45c1-a74d-47fd0378bb59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:41.469769 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:41.534777 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6280619df0e4dac84868ea94e7885e8', 'request_id': 'req-843286f5-f23b-45c1-a74d-47fd0378bb59'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:41.551659 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] The token cache is being invalidate because project f6280619df0e4dac84868ea94e7885e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:40:41.552154 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:41.552967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:41.554739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1402/2778] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:40:41 2022] DELETE /identity/v3/projects/f6280619df0e4dac84868ea94e7885e8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:40:50.949335 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:51.656021 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=1,keystone:SELECT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:40:55.981386 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:55.981888 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:55.982013 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:55.999874 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] MFA Rules not processed for user `c58ac60d4c7441a4ba1053cbbb14b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:40:56.001087 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:56.056199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1377/2779] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jun 1 04:40:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:40:56.070006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:56.075441 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:56.142391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843286f5-f23b-45c1-a74d-47fd0378bb59 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:56.147416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '400527de384349f58866dd9e39468307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:56.148076 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:56.148405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:56.148643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:56.157692 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:56.159270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:56.167723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1403/2780] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jun 1 04:40:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 01 04:40:59.020662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:59.022215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:59.094205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7218f-638f-45e5-a679-4fa9fea23ce3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:59.099217 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:59.100348 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:59.100632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:59.100942 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] PATH_INFO: `/v3/users/80b97ad4fa674391a99cf97bc10da8bc` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:59.102163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b97ad4fa674391a99cf97bc10da8bc)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:59.104273 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:59.289048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b97ad4fa674391a99cf97bc10da8bc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.296104 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b97ad4fa674391a99cf97bc10da8bc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.296822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b97ad4fa674391a99cf97bc10da8bc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.310278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b97ad4fa674391a99cf97bc10da8bc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.385565 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e082628-ed01-4a85-8398-a06843a0231b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b97ad4fa674391a99cf97bc10da8bc'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.392733 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1378/2781] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:40:59 2022] DELETE /identity/v3/users/80b97ad4fa674391a99cf97bc10da8bc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:40:59.644585 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:40:59.666340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d2515-e954-464a-b884-6beafaf5d8eb service swift] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:40:59.712016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:40:59.712882 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:59.713164 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:59.713478 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] PATH_INFO: `/v3/projects/6607408c2a46439d93026002d312b9de` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:59.714657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6607408c2a46439d93026002d312b9de)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:40:59.717436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:40:59.831325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6607408c2a46439d93026002d312b9de', 'request_id': 'req-4b19a930-4f55-4ee8-9711-370631268710'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.837018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6607408c2a46439d93026002d312b9de', 'request_id': 'req-4b19a930-4f55-4ee8-9711-370631268710'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.905272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6607408c2a46439d93026002d312b9de', 'request_id': 'req-4b19a930-4f55-4ee8-9711-370631268710'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.916663 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] The token cache is being invalidate because project 6607408c2a46439d93026002d312b9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:40:59.918151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.918151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-4b19a930-4f55-4ee8-9711-370631268710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:40:59.919723 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1404/2782] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:40:59 2022] DELETE /identity/v3/projects/6607408c2a46439d93026002d312b9de => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:40:59.924822 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:40:59.925117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:40:59.925395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:40:59.954342 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:40:59.955590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:40:59.964783 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:00.013405 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1379/2783] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:41:00.021349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.021645 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.021931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.040105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:41:00.041302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:00.049742 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1405/2784] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:41:00.054109 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.064620 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208c65-243d-4bd6-a709-85e6860dc0e2 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.069755 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.070415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.070700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.071004 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.071764 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:41:00.072077 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.073685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.085078 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1380/2785] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:41:00.089505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.098790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b9c94-96b8-4894-b172-f7b9b7b4068b None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.103553 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.104201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.104748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.105029 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.105730 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.108178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.117881 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:00.206141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1406/2786] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:41:00 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:41:00.207614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.217936 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2acd14-c468-49dd-9dae-14bfb24a118b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.272996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.273659 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.273945 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.274221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.274910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.276846 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.355729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1381/2787] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:41:00 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:41:00.360558 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.370980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4cce-5411-4392-a315-0d74adadc0a3 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.375587 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.376236 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.376518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.376797 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.377514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.379163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.389504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1407/2788] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:41:00.393365 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.403778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b718f-f6ae-46c6-b852-563ea65cba66 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.408359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2006e5-55ec-456d-99dd-0b2715c65c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.409010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2006e5-55ec-456d-99dd-0b2715c65c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.409295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2006e5-55ec-456d-99dd-0b2715c65c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.409570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2006e5-55ec-456d-99dd-0b2715c65c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.410291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2006e5-55ec-456d-99dd-0b2715c65c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.411904 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2006e5-55ec-456d-99dd-0b2715c65c5d admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.422316 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1382/2789] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:41:00.426291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.436116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b44a-adbc-4b1f-bd97-6d3cf71c55f3 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.440725 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.441599 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.441885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.442166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] PATH_INFO: `/v3/projects/d30fc2f66738421ea107f0e4a1fd9c14/users/4c8c18c552614f8abfa755360b714b21/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.456511 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30fc2f66738421ea107f0e4a1fd9c14, user_id=4c8c18c552614f8abfa755360b714b21, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.459100 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.567657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1408/2790] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:41:00 2022] PUT /identity/v3/projects/d30fc2f66738421ea107f0e4a1fd9c14/users/4c8c18c552614f8abfa755360b714b21/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:41:00.578913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6e151-4892-4d0d-8459-e27dfe7ae21c None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.579288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6e151-4892-4d0d-8459-e27dfe7ae21c None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.579603 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6e151-4892-4d0d-8459-e27dfe7ae21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.606694 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-15d6e151-4892-4d0d-8459-e27dfe7ae21c None None] MFA Rules not processed for user `4c8c18c552614f8abfa755360b714b21`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:41:00.607924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-15d6e151-4892-4d0d-8459-e27dfe7ae21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:00.672065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1383/2791] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:41:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:41:00.687254 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:00.689412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:00.780134 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38dda1-74e3-4613-a73a-b9edc33d6fbf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:00.783847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:00.784503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:00.784788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:00.785066 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:00.794075 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:00.795555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:00.804646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1409/2792] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:41:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:41:10.669732 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:10.800405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:19.392306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:19.396091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:19.396091 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:19.411643 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] MFA Rules not processed for user `a17f39a1626d45299666f43be4ffc3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:41:19.412837 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:19.468371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1384/2793] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:41:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:41:19.782989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:19.784486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:19.842818 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3648a6-58fb-4f3c-8506-5589e294545d service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:19.846731 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:19.847402 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:19.847694 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:19.847979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:19.849833 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:19.861953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:19.863482 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:19.871819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1410/2794] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:41:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:41:29.460980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=25 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:29.868194 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=26 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:43.814781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:43.816193 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:43.910150 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13803ad-8924-4bcc-a81e-00433e7a4272 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:43.918429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:43.919107 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:43.919395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:43.919675 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:43.921972 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:43.934327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:43.935980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871f402-e440-4853-a484-82e4b60c4c76 service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:43.946112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1385/2795] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:41:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:41:46.668567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:46.732179 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62de79-ad3a-404b-a3d6-2b989eb383da service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:46.737386 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:46.738289 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:46.738569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:46.738841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] PATH_INFO: `/v3/users/4c8c18c552614f8abfa755360b714b21` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:46.740040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c8c18c552614f8abfa755360b714b21)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:46.742123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:46.896184 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c8c18c552614f8abfa755360b714b21'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:46.896931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c8c18c552614f8abfa755360b714b21'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:46.910195 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c8c18c552614f8abfa755360b714b21'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:46.917824 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c8c18c552614f8abfa755360b714b21'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:46.990518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c8c18c552614f8abfa755360b714b21'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:46.997057 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1411/2796] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:41:46 2022] DELETE /identity/v3/users/4c8c18c552614f8abfa755360b714b21 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:41:47.001695 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:47.002184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:47.002306 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:47.020399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:41:47.021698 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:47.080890 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1386/2797] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:41:47.090819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:47.092841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:47.149563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa870b-3100-4774-8546-173949484892 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:47.153083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:47.154319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:47.154319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:47.154319 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:47.164310 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:47.165902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:47.175321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1412/2798] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:41:47.791363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:41:47.792789 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:41:47.805269 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93041-bc7c-4aab-a175-ab0dce118441 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:41:47.809873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:41:47.810750 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:41:47.811623 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:41:47.811903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] PATH_INFO: `/v3/projects/d30fc2f66738421ea107f0e4a1fd9c14` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:41:47.813138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30fc2f66738421ea107f0e4a1fd9c14)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:41:47.815153 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:41:47.916468 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30fc2f66738421ea107f0e4a1fd9c14', 'request_id': 'req-f84054ff-36d9-49d2-a121-716cc98e54e3'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:47.989857 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30fc2f66738421ea107f0e4a1fd9c14', 'request_id': 'req-f84054ff-36d9-49d2-a121-716cc98e54e3'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:47.995774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30fc2f66738421ea107f0e4a1fd9c14', 'request_id': 'req-f84054ff-36d9-49d2-a121-716cc98e54e3'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:47.996883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:48.007873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] The token cache is being invalidate because project d30fc2f66738421ea107f0e4a1fd9c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:41:48.008461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:48.008983 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:41:48.010512 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1387/2799] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:41:47 2022] DELETE /identity/v3/projects/d30fc2f66738421ea107f0e4a1fd9c14 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:41:57.171145 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:41:58.144177 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:42:39.392803 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:42:39.401594 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba6950-0a59-436c-887e-884f05dd0af2 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:42:39.454674 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:42:39.455366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:39.455838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:39.455979 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:39.457819 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:39.542884 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:42:39.544375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136a78c-4d6b-4ad9-841e-4aaf7f042389 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:42:39.565333 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1413/2800] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:42:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:42:40.747421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:42:40.748852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:40.761098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84054ff-36d9-49d2-a121-716cc98e54e3 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:42:40.764812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:42:40.765448 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:40.765730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:40.766010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:40.767906 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:40.781811 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:42:40.783231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a839-1a3e-48e4-9488-aaea6d81ac2f service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:42:40.794587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1388/2801] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:42:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:42:49.562717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=63 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:42:50.789084 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=6 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:42:55.483155 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b69a3-fede-4acc-bf86-b5f4dffc5284 None None] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:55.484217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b69a3-fede-4acc-bf86-b5f4dffc5284 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:55.484408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b69a3-fede-4acc-bf86-b5f4dffc5284 None None] PATH_INFO: `/` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:55.485810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1414/2802] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:42:55.491660 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd0180-fae9-4736-b1a6-13d5ffe904b0 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:55.492108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd0180-fae9-4736-b1a6-13d5ffe904b0 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:55.492248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd0180-fae9-4736-b1a6-13d5ffe904b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:55.533164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-10bd0180-fae9-4736-b1a6-13d5ffe904b0 None None] MFA Rules not processed for user `885cc3a3c23c451c9ad7526e7fa83639`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:42:55.534392 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-10bd0180-fae9-4736-b1a6-13d5ffe904b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:55.540969 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1389/2803] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:42:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:42:55.553522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b69a3-fede-4acc-bf86-b5f4dffc5284 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:42:55.555266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-cb9b69a3-fede-4acc-bf86-b5f4dffc5284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:55.568908 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b69a3-fede-4acc-bf86-b5f4dffc5284 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:42:55.572410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4efd27-870a-405f-ae1b-1f7661f18178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:42:55.573037 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4efd27-870a-405f-ae1b-1f7661f18178 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:55.573320 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4efd27-870a-405f-ae1b-1f7661f18178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:55.573597 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4efd27-870a-405f-ae1b-1f7661f18178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:55.581191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4efd27-870a-405f-ae1b-1f7661f18178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:42:55.582622 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4efd27-870a-405f-ae1b-1f7661f18178 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:42:55.590323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1415/2804] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:42:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:42:55.675336 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49789b33-5969-455d-97ac-79a3e07f274e None None] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:55.675641 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49789b33-5969-455d-97ac-79a3e07f274e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:55.675929 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49789b33-5969-455d-97ac-79a3e07f274e None None] PATH_INFO: `/` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:55.677460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1390/2805] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jun 1 04:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 01 04:42:55.682821 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:55.683116 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:55.683504 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:55.719959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] MFA Rules not processed for user `56036136066d412dac682273ebf65ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:42:55.721152 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:55.730572 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1416/2806] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jun 1 04:42:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:42:55.740976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49789b33-5969-455d-97ac-79a3e07f274e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:42:55.752478 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49789b33-5969-455d-97ac-79a3e07f274e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:42:55.756011 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848dc7f-1a5a-4535-ac23-861c06b791a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:42:55.756637 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848dc7f-1a5a-4535-ac23-861c06b791a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:55.756918 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848dc7f-1a5a-4535-ac23-861c06b791a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:55.757194 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848dc7f-1a5a-4535-ac23-861c06b791a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:55.767758 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848dc7f-1a5a-4535-ac23-861c06b791a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:42:55.769184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848dc7f-1a5a-4535-ac23-861c06b791a8 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:42:55.779654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1391/2807] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:42:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:42:58.751662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:42:58.761378 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6933-c8af-4d48-bbd6-b8aa3efc7cb4 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:42:58.764907 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:42:58.765539 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:42:58.765822 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:42:58.766141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:42:58.767951 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:42:58.777436 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:42:58.778862 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:42:58.787372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1417/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:42:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:43:05.775068 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=15 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:43:07.515652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:43:07.516339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:43:07.516339 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:43:07.533774 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] MFA Rules not processed for user `a17f39a1626d45299666f43be4ffc3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:43:07.534959 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:43:07.594363 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1392/2809] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:43:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:43:08.783714 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=21 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:43:17.587276 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=25 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:44:39.818071 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:39.821685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:39.836026 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204e7c9-3e4c-4f95-ae54-158390064a17 service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:39.841434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:39.844653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:39.844653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:39.844653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:39.845096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:39.855703 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:39.857332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:39.866211 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1418/2810] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:44:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:44:42.771148 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:42.781041 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ad5e-7673-4cd3-90fb-ea441f33abc4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:42.784852 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:42.785526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:42.785814 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:42.786095 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:42.788017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:42.850561 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:42.852060 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:42.861061 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1393/2811] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:44:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:44:43.817025 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:43.818761 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:43.835617 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0068e2-bfd8-4fba-b0a5-90ead7e53f8c service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:43.841522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:43.842988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:43.842988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:43.842988 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:43.845486 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:43.862040 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:43.867410 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:43.876838 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1419/2812] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:44:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:44:49.211933 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:49.221889 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018b146-0da7-4b9a-8dc2-1c6ab69ada79 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:49.225781 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:49.226424 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:49.226713 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:49.226995 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:49.228818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:49.291033 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:49.292542 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:49.302356 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1394/2813] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:44:53.249743 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:53.252326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:53.347265 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32dfa1-726c-4839-86e3-84b1bbb1b1aa service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:53.361272 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:53.361711 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:53.362006 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:53.362288 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:53.367518 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:53.382001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:53.383900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:53.395408 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1420/2814] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:44:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:44:57.148247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:57.150018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:57.164211 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de80c7d-9e43-4235-aa8e-ab09b27e6525 service neutron] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:57.169018 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:57.170083 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:57.170368 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:57.170664 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] PATH_INFO: `/v3/users/44b4ed73a96143098d2c64384233d539` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:57.171869 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b4ed73a96143098d2c64384233d539)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:57.173794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:57.312400 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b4ed73a96143098d2c64384233d539'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:57.319554 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b4ed73a96143098d2c64384233d539'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:57.320230 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b4ed73a96143098d2c64384233d539'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:57.333309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b4ed73a96143098d2c64384233d539'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:57.406836 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b4ed73a96143098d2c64384233d539'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:57.413525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1395/2815] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:44:57 2022] DELETE /identity/v3/users/44b4ed73a96143098d2c64384233d539 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:44:57.708041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:57.717471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c71cc7-f048-442b-83f1-300cc2c7101a service placement] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:57.793902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:57.794772 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:57.795063 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:57.795505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] PATH_INFO: `/v3/projects/6d70e82d96624016bed251b30b26ad73` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:57.801349 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d70e82d96624016bed251b30b26ad73)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:57.804844 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:57.930086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d70e82d96624016bed251b30b26ad73', 'request_id': 'req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:57.940058 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d70e82d96624016bed251b30b26ad73', 'request_id': 'req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.015561 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d70e82d96624016bed251b30b26ad73', 'request_id': 'req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.018583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:44:58.038048 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] The token cache is being invalidate because project 6d70e82d96624016bed251b30b26ad73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:44:58.038352 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.039034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.040972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1421/2816] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:44:57 2022] DELETE /identity/v3/projects/6d70e82d96624016bed251b30b26ad73 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:44:58.046818 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:58.048135 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:58.060577 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:44:58.065235 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e5cb-fe9b-4f1f-9879-7e89ebd1273b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:58.149023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:58.152072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:58.152072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:58.152072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] PATH_INFO: `/v3/users/b644b980149f4ae4a1b469ea78a8718a` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:58.152072 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b644b980149f4ae4a1b469ea78a8718a)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:58.154184 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:58.380873 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b644b980149f4ae4a1b469ea78a8718a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.388520 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b644b980149f4ae4a1b469ea78a8718a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.389278 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b644b980149f4ae4a1b469ea78a8718a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.407686 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b644b980149f4ae4a1b469ea78a8718a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.498288 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f27df1c5-f7c0-4836-a1ef-b5207035e123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b644b980149f4ae4a1b469ea78a8718a'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:58.505048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1396/2817] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:44:58 2022] DELETE /identity/v3/users/b644b980149f4ae4a1b469ea78a8718a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:44:58.773606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:44:58.776307 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:44:58.801323 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4af4b5-0b5f-4d71-abd3-55dbfbfeb4b1 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:44:58.882957 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:44:58.884465 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:44:58.884760 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:44:58.885304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] PATH_INFO: `/v3/projects/e02b170043e14f41923636805f4e2a15` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:44:58.886767 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02b170043e14f41923636805f4e2a15)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:44:58.889089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:44:59.039649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02b170043e14f41923636805f4e2a15', 'request_id': 'req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:59.047104 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02b170043e14f41923636805f4e2a15', 'request_id': 'req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:59.173466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02b170043e14f41923636805f4e2a15', 'request_id': 'req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:59.187443 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] The token cache is being invalidate because project e02b170043e14f41923636805f4e2a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:44:59.187757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:59.188437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:44:59.189978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1422/2818] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:44:58 2022] DELETE /identity/v3/projects/e02b170043e14f41923636805f4e2a15 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:45:03.409980 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:45:03.410498 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:45:03.410611 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:45:03.435778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] MFA Rules not processed for user `a17f39a1626d45299666f43be4ffc3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:45:03.435778 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:03.492224 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1397/2819] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:45:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:45:04.151938 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:45:04.157865 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:04.231113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923c32b-d7d8-464a-b9e6-0e84a73b3c4a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:45:04.236316 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:45:04.236962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:45:04.237247 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:45:04.237527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:45:04.239335 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:04.251034 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:45:04.252632 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:45:04.262690 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1423/2820] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:45:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:45:13.485128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=76,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:45:14.258230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:45:15.374108 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:45:15.378309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:15.448325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78393e-05d6-4053-b0d7-f0f110dac077 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:45:15.454311 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:45:15.454971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:45:15.455254 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:45:15.455560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:45:15.457474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:15.470485 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:45:15.472098 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:45:15.483067 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1398/2821] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:45:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:45:19.134508 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:45:19.135965 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:19.151547 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ccab2-ad86-43df-9703-1d5e83d3a0c7 service placement] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:45:19.157648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56036136066d412dac682273ebf65ab9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:45:19.158305 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:45:19.158596 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:45:19.158878 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:45:19.160707 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:45:19.173903 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:45:19.175529 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:45:19.186432 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1424/2822] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:45:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:45:25.478619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=34 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:45:29.181748 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=6 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:46:47.771117 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:47.772591 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:47.786900 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914bd02-e53a-44bf-bb38-72e87be2bf82 service nova] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:47.792029 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:47.792680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:47.792965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:47.793243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:47.795580 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:47.811730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:47.811730 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:47.824382 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1399/2823] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:46:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:46:54.368734 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:54.370178 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:54.425567 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a7e98-f7ea-4b62-9731-ef2d5ea47c24 service nova] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:54.429119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:54.429756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:54.430038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:54.430313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:54.432094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:54.494363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:54.495848 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:54.506046 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1425/2824] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:46:57.820759 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=6 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:46:58.188031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:58.192507 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:58.205457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb7375-0cf9-4019-9e9c-23c0e0213332 service placement] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:58.210130 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:58.211017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:58.211325 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:58.211619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] PATH_INFO: `/v3/users/a17f39a1626d45299666f43be4ffc3ec` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:58.212905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17f39a1626d45299666f43be4ffc3ec)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:58.214910 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:58.450391 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17f39a1626d45299666f43be4ffc3ec'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:58.457866 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17f39a1626d45299666f43be4ffc3ec'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:58.458543 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17f39a1626d45299666f43be4ffc3ec'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:58.471556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17f39a1626d45299666f43be4ffc3ec'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:58.553521 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-ea94bbcb-fbfa-46b2-82dd-35415ea501b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17f39a1626d45299666f43be4ffc3ec'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:58.560978 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1400/2825] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:46:58 2022] DELETE /identity/v3/users/a17f39a1626d45299666f43be4ffc3ec => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:46:58.864035 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:58.877327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c813f4-17f7-49cc-8f3a-cc0121d5b17d service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:58.937754 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:58.938646 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:58.939005 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:58.939192 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] PATH_INFO: `/v3/projects/756cb4ed8d2345dcaae27638b257a480` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:58.940412 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] RBAC: Authorizing `identity:delete_project(project_id=756cb4ed8d2345dcaae27638b257a480)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:58.943366 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.058497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '756cb4ed8d2345dcaae27638b257a480', 'request_id': 'req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:59.061054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:46:59.064937 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '756cb4ed8d2345dcaae27638b257a480', 'request_id': 'req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:59.136270 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '756cb4ed8d2345dcaae27638b257a480', 'request_id': 'req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:59.150163 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] The token cache is being invalidate because project 756cb4ed8d2345dcaae27638b257a480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80710) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:46:59.150710 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:59.151774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ccacaf3f-56c3-4d21-bf20-6a429b53ebee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:46:59.154010 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1426/2826] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:46:58 2022] DELETE /identity/v3/projects/756cb4ed8d2345dcaae27638b257a480 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:46:59.159574 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.160131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.160131 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.180652 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:46:59.187808 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:59.245720 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1401/2827] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 01 04:46:59.253953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.254419 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.254897 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.275610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:46:59.276835 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:59.285578 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1427/2828] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:46:59.289913 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:59.300549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c93f3b-79ce-4eeb-888f-aacd96e0e21e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:59.305189 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:59.305861 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.306152 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.306462 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] PATH_INFO: `/v3/domains` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.307200 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:46:59.307549 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:59.309223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.321389 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1402/2829] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:46:59.325841 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:59.335411 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534964a9-e3e8-43df-a69e-5777098b70c0 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:59.340252 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:59.340900 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.341181 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.341455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.342650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:59.344697 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.445776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1428/2830] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:46:59.450090 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:59.463323 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ade3f-c300-4571-b06a-a7d5a850a324 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:59.524701 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:59.525397 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.525679 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.525960 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.526655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:59.528614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.536007 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:46:59.679247 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1403/2831] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:46:59 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:46:59.683740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:59.694365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cb2ae-4459-47b2-b50f-a337d2ba2faf admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:59.699041 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:59.699768 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.703827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.703827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.703827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:59.703827 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.713825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1429/2832] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:46:59.718578 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:59.733439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354b16a-550e-493f-a41e-09b41552ee97 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:59.738213 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16d4ae-7c1d-4c91-8782-ed601d162816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:59.738884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16d4ae-7c1d-4c91-8782-ed601d162816 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.739170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16d4ae-7c1d-4c91-8782-ed601d162816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.739493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16d4ae-7c1d-4c91-8782-ed601d162816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.740292 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16d4ae-7c1d-4c91-8782-ed601d162816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:59.741898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16d4ae-7c1d-4c91-8782-ed601d162816 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.753358 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1404/2833] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:46:59.757479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:46:59.768343 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912d904-e689-4290-bf20-e6f3c7ab045b admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:46:59.773105 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:46:59.773989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.774274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.774579 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] PATH_INFO: `/v3/projects/f331273530a74af48862e4c4ba0a7634/users/d5a425cd150f403dbb4fd3f7fccb121f/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.790223 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f331273530a74af48862e4c4ba0a7634, user_id=d5a425cd150f403dbb4fd3f7fccb121f, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:46:59.792966 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:46:59.861151 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1430/2834] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:46:59 2022] PUT /identity/v3/projects/f331273530a74af48862e4c4ba0a7634/users/d5a425cd150f403dbb4fd3f7fccb121f/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:46:59.871262 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:46:59.871622 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:46:59.871923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:46:59.899267 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] MFA Rules not processed for user `d5a425cd150f403dbb4fd3f7fccb121f`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:46:59.900472 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:46:59.965505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1405/2835] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:46:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:46:59.970332 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.033141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76d5a-8e53-4114-b099-a05e9fdb4493 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.038028 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.038685 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.038973 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.039251 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.040001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.041901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:00.149860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1431/2836] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:46:59 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:00.153812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.165074 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cdf59-4c3d-4c86-9983-984e99843db4 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.228242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.228923 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.229210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.229488 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.230183 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.232141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:00.342290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1406/2837] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:47:00 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:00.346350 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.357160 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c4dc-823c-4261-a0b0-be2c63441303 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.362978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.363683 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.363967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.364268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.364991 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.366592 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:00.378837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1432/2838] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:00.382976 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.393281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e865c2-c73b-4b46-8241-351559cdf904 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.398066 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f092445-4570-461c-90e3-8f8a922dadb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.398752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f092445-4570-461c-90e3-8f8a922dadb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.399155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f092445-4570-461c-90e3-8f8a922dadb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.399471 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f092445-4570-461c-90e3-8f8a922dadb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.400287 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f092445-4570-461c-90e3-8f8a922dadb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.401829 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f092445-4570-461c-90e3-8f8a922dadb7 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:00.413390 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1407/2839] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:00.417812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.429959 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb5ed7-1566-4857-8e11-27106da247d2 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.434860 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.435783 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.436712 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.436989 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] PATH_INFO: `/v3/projects/a30af4c2b02c4e1e8c9c51dd56107d05/users/001a08b90e444d40b73efc9c5401d64a/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.454978 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30af4c2b02c4e1e8c9c51dd56107d05, user_id=001a08b90e444d40b73efc9c5401d64a, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.457888 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:00.590086 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1433/2840] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:00 2022] PUT /identity/v3/projects/a30af4c2b02c4e1e8c9c51dd56107d05/users/001a08b90e444d40b73efc9c5401d64a/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:00.602525 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.602817 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.603162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.632493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] MFA Rules not processed for user `001a08b90e444d40b73efc9c5401d64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:00.633707 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:00.674665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:00.713140 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1408/2841] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:00.719045 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.740963 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:00.786141 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c4a0a-396a-4aeb-9dca-624886d0ae98 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.791434 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.792102 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.792389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.792671 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.793365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.795856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:00.909679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1434/2842] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:47:00 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:00.913187 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:00.925138 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ff268-0146-431e-8b91-75ec8c8a0902 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:00.981648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:00.982318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:00.982710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:00.983538 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] PATH_INFO: `/v3/users` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:00.984225 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:00.986142 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:01.091436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1409/2843] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:47:00 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:01.095926 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:01.109080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081df35-db02-4b03-af3d-792f8c090a61 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:01.113871 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:01.126327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:01.126327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:01.126327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:01.126327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:01.126327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:01.128498 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1435/2844] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:01.132794 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:01.143926 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7e96-9849-452b-baa2-1c5dc75ccd47 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:01.148722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:01.149994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:01.149994 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:01.150464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] PATH_INFO: `/v3/projects/4a2ce4fcdefa484bac858a292204d6bb/users/b3ea9eee3c1847de8764293296706031/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:01.166824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2ce4fcdefa484bac858a292204d6bb, user_id=b3ea9eee3c1847de8764293296706031, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:01.169556 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:01.262824 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1410/2845] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:01 2022] PUT /identity/v3/projects/4a2ce4fcdefa484bac858a292204d6bb/users/b3ea9eee3c1847de8764293296706031/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:01.267360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:01.277569 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9acc9-ef45-4138-a345-f88ad635a8d9 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:01.335089 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e466604-5eb7-4ee8-bf63-f09d99c650e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:01.335795 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e466604-5eb7-4ee8-bf63-f09d99c650e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:01.336119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e466604-5eb7-4ee8-bf63-f09d99c650e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:01.336394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e466604-5eb7-4ee8-bf63-f09d99c650e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:01.337121 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e466604-5eb7-4ee8-bf63-f09d99c650e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:01.338776 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e466604-5eb7-4ee8-bf63-f09d99c650e9 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:01.349855 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1436/2846] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:01.354017 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:01.364634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19bcf-0113-4361-b014-811ac7bb7b18 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:01.372831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:01.372831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:01.372831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:01.372831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] PATH_INFO: `/v3/projects/4a2ce4fcdefa484bac858a292204d6bb/users/b3ea9eee3c1847de8764293296706031/roles/02d9e6449ea143beb203824112640698` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:01.373802 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2ce4fcdefa484bac858a292204d6bb, user_id=b3ea9eee3c1847de8764293296706031, role_id=02d9e6449ea143beb203824112640698)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:01.375965 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:01.386532 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1411/2847] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:01 2022] PUT /identity/v3/projects/4a2ce4fcdefa484bac858a292204d6bb/users/b3ea9eee3c1847de8764293296706031/roles/02d9e6449ea143beb203824112640698 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:01.397220 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:01.397516 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:01.397873 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:01.424854 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] MFA Rules not processed for user `b3ea9eee3c1847de8764293296706031`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:01.426166 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:01.503982 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1437/2848] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:01.515349 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:01.516982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:01.581639 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759f37e-915e-4600-bd94-0e75ae19a368 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:01.585299 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:01.586002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:01.586290 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:01.586571 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:01.596050 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:01.597765 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:01.607381 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1412/2849] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:02.724851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:02.726282 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:02.739123 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee62179-81b5-41c5-8c9b-f3228acf5aab None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:02.742932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:02.743643 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:02.743932 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:02.744212 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:02.754326 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:02.801667 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:02.803130 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:02.812732 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1438/2850] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:08.864481 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:08.873715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a852d5-b8a4-4d6c-b03e-ccea77faf9c7 service cinder] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:08.878196 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563d1be-e19a-4fa8-a264-1c98214b58c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:08.878844 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563d1be-e19a-4fa8-a264-1c98214b58c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:08.879120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563d1be-e19a-4fa8-a264-1c98214b58c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:08.879439 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563d1be-e19a-4fa8-a264-1c98214b58c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:08.909457 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:08.937905 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563d1be-e19a-4fa8-a264-1c98214b58c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:08.939371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563d1be-e19a-4fa8-a264-1c98214b58c4 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:08.948353 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1413/2851] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 01 04:47:12.861619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=22 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:13.127294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:13.190294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885825a4-8b75-405d-a699-bee3332147ce service cinder] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:13.195505 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:13.196371 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:13.196656 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:13.196936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] PATH_INFO: `/v3/users/d5a425cd150f403dbb4fd3f7fccb121f` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:13.198120 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a425cd150f403dbb4fd3f7fccb121f)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:13.200230 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:13.383514 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a425cd150f403dbb4fd3f7fccb121f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:13.384099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a425cd150f403dbb4fd3f7fccb121f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:13.396264 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a425cd150f403dbb4fd3f7fccb121f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:13.403210 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a425cd150f403dbb4fd3f7fccb121f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:13.467001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a425cd150f403dbb4fd3f7fccb121f'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:13.474555 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1439/2852] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:13 2022] DELETE /identity/v3/users/d5a425cd150f403dbb4fd3f7fccb121f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:13.479291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:13.479835 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:13.479950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:13.498674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:13.499950 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:13.558598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1414/2853] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:13.568704 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:13.570119 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:13.631808 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e62950-9c39-4aed-b6c9-aa003115ada7 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:13.635493 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:13.636080 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:13.636360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:13.636635 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:13.646381 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:13.647851 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:13.658093 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1440/2854] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:14.069479 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:14.071586 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:14.083651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa3449-3827-4e98-9b66-c964202d75a3 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:14.089143 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:14.090038 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:14.090437 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:14.090729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] PATH_INFO: `/v3/projects/f331273530a74af48862e4c4ba0a7634` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:14.092830 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f331273530a74af48862e4c4ba0a7634)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:14.094955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:14.198667 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f331273530a74af48862e4c4ba0a7634', 'request_id': 'req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.272021 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f331273530a74af48862e4c4ba0a7634', 'request_id': 'req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.277842 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f331273530a74af48862e4c4ba0a7634', 'request_id': 'req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.288951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] The token cache is being invalidate because project f331273530a74af48862e4c4ba0a7634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:14.289319 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.289903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.291493 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1415/2855] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:14 2022] DELETE /identity/v3/projects/f331273530a74af48862e4c4ba0a7634 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:14.295577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:14.305400 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337764f4-89d7-44ba-b15f-f66b409e59cd service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:14.324292 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:14.362027 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:14.362985 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:14.363273 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:14.364099 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] PATH_INFO: `/v3/users/001a08b90e444d40b73efc9c5401d64a` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:14.365274 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=001a08b90e444d40b73efc9c5401d64a)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:14.367241 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:14.582901 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001a08b90e444d40b73efc9c5401d64a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.583600 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001a08b90e444d40b73efc9c5401d64a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.595648 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001a08b90e444d40b73efc9c5401d64a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.602885 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001a08b90e444d40b73efc9c5401d64a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.678654 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001a08b90e444d40b73efc9c5401d64a'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:14.685098 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1441/2856] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:14 2022] DELETE /identity/v3/users/001a08b90e444d40b73efc9c5401d64a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:15.067648 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:15.069544 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:15.084371 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e750651-92d0-4fb8-9ad5-b7b24a32dcb1 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:15.134697 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:15.143257 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:15.144173 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:15.144484 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:15.145410 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] PATH_INFO: `/v3/projects/a30af4c2b02c4e1e8c9c51dd56107d05` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:15.146609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30af4c2b02c4e1e8c9c51dd56107d05)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:15.148581 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:15.256872 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30af4c2b02c4e1e8c9c51dd56107d05', 'request_id': 'req-346cf342-5812-46b6-accc-8d8dfa660138'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.324409 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30af4c2b02c4e1e8c9c51dd56107d05', 'request_id': 'req-346cf342-5812-46b6-accc-8d8dfa660138'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.330434 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30af4c2b02c4e1e8c9c51dd56107d05', 'request_id': 'req-346cf342-5812-46b6-accc-8d8dfa660138'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.348672 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] The token cache is being invalidate because project a30af4c2b02c4e1e8c9c51dd56107d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:15.348971 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.349608 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.351202 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1416/2857] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:15 2022] DELETE /identity/v3/projects/a30af4c2b02c4e1e8c9c51dd56107d05 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:15.356239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:15.356696 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:15.369483 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cd69e-c7d6-4cf4-b9e5-b423e1fb0c41 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:15.431956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:15.432740 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:15.433019 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:15.433291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] PATH_INFO: `/v3/users/b3ea9eee3c1847de8764293296706031` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:15.434469 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ea9eee3c1847de8764293296706031)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:15.436608 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:15.741348 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:15.743980 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ea9eee3c1847de8764293296706031'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.744634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ea9eee3c1847de8764293296706031'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.756933 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ea9eee3c1847de8764293296706031'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.763560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ea9eee3c1847de8764293296706031'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.833055 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-d87e4514-c799-4cab-b3ec-392a9c0ea614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ea9eee3c1847de8764293296706031'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:15.839363 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1442/2858] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:15 2022] DELETE /identity/v3/users/b3ea9eee3c1847de8764293296706031 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:16.216941 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:16.218403 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:16.232242 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf342-5812-46b6-accc-8d8dfa660138 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:16.291585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:16.292377 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:16.292662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:16.292937 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] PATH_INFO: `/v3/projects/4a2ce4fcdefa484bac858a292204d6bb` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:16.294266 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2ce4fcdefa484bac858a292204d6bb)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:16.296199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:16.433850 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2ce4fcdefa484bac858a292204d6bb', 'request_id': 'req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:16.558760 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2ce4fcdefa484bac858a292204d6bb', 'request_id': 'req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:16.565255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2ce4fcdefa484bac858a292204d6bb', 'request_id': 'req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:16.580588 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] The token cache is being invalidate because project 4a2ce4fcdefa484bac858a292204d6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:16.580885 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:16.581562 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-cfb520a3-a1f7-4eaf-bed2-b3c57245063b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:16.583838 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1417/2859] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:16 2022] DELETE /identity/v3/projects/4a2ce4fcdefa484bac858a292204d6bb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:16.589956 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:16.590545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:16.590545 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:16.611610 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:16.612839 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:16.681144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1443/2860] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:16.690881 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:16.691114 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:16.691429 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:16.712777 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:16.714012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:16.725089 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1418/2861] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:16.729346 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:16.738954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02304f-7e09-46c2-9b80-69e2e80ef642 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:16.743688 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:16.744372 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:16.744653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:16.744927 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:16.745649 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:47:16.745953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:16.747631 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:16.761528 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1444/2862] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:47:16.765996 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:16.775871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046de95-18f0-442e-8f1f-bede1225ab92 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:16.780632 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:16.781289 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:16.781572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:16.781849 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:16.782541 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:16.784461 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:16.788570 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:16.897880 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1419/2863] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:47:16 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:47:16.902083 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:16.912836 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0491-93cd-4c13-93d0-518d090f5174 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:16.972340 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:16.973634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:16.973914 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:16.974191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:16.974947 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:16.976902 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:17.069805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1445/2864] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:47:16 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:17.073970 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:17.085480 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb8a3f-846c-48d1-8231-d06ab86f7c8f admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:17.090729 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:17.091442 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:17.091740 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:17.092020 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:17.092767 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:17.094495 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:17.105413 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1420/2865] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:17.110266 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:17.122405 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53a1eb-152d-474d-8d37-e23850a73cf8 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:17.127036 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62721c-72ce-4559-bc40-f52eb69889cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:17.128259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62721c-72ce-4559-bc40-f52eb69889cb admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:17.128259 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62721c-72ce-4559-bc40-f52eb69889cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:17.128522 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62721c-72ce-4559-bc40-f52eb69889cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:17.129771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62721c-72ce-4559-bc40-f52eb69889cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:17.131395 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62721c-72ce-4559-bc40-f52eb69889cb admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:17.142278 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1446/2866] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:17.150120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:17.160505 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066550e1-984a-440d-8190-14c30619ab12 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:17.165263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:17.166159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:17.166440 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:17.166715 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] PATH_INFO: `/v3/projects/d1d04dbac9284e91911b5ad8fc1bbd5b/users/289bddd0e73c480383ac26c98b36fdc0/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:17.182291 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d04dbac9284e91911b5ad8fc1bbd5b, user_id=289bddd0e73c480383ac26c98b36fdc0, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:17.184997 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:17.284334 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1421/2867] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:17 2022] PUT /identity/v3/projects/d1d04dbac9284e91911b5ad8fc1bbd5b/users/289bddd0e73c480383ac26c98b36fdc0/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:17.348753 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:17.349022 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:17.349300 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:17.382237 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] MFA Rules not processed for user `289bddd0e73c480383ac26c98b36fdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:17.383510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:17.396563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:17.455404 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1447/2868] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:17.467064 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:17.468951 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:17.530826 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdf939-92dc-49b4-b87f-214baa0d0700 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:17.534522 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9f157-7da5-4686-a7df-1c6d038901c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:17.535178 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f157-7da5-4686-a7df-1c6d038901c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:17.535490 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f157-7da5-4686-a7df-1c6d038901c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:17.535768 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9f157-7da5-4686-a7df-1c6d038901c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:17.545205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9f157-7da5-4686-a7df-1c6d038901c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:17.546831 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9f157-7da5-4686-a7df-1c6d038901c6 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:17.557199 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1422/2869] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:27.506960 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=20 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:27.552762 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:28.613746 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:28.674913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39b4f9-a7c9-4c11-8dd5-5e9f6670a1ef None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:28.680201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:28.681133 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:28.681416 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:28.681747 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] PATH_INFO: `/v3/users/289bddd0e73c480383ac26c98b36fdc0` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:28.682934 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=289bddd0e73c480383ac26c98b36fdc0)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:28.684911 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:28.897324 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289bddd0e73c480383ac26c98b36fdc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:28.897984 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289bddd0e73c480383ac26c98b36fdc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:28.910627 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289bddd0e73c480383ac26c98b36fdc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:28.917935 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289bddd0e73c480383ac26c98b36fdc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.013972 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289bddd0e73c480383ac26c98b36fdc0'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.020280 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1448/2870] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:28 2022] DELETE /identity/v3/users/289bddd0e73c480383ac26c98b36fdc0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:47:29.025393 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:29.029597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:29.029597 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:29.044207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:29.045395 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:29.102674 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1423/2871] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:29.112228 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:29.113719 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:29.170857 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faa407-49c3-4763-98c4-05ba48e4e47a admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:29.174325 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e589-186d-4c13-a092-9ed53dea91aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:29.174970 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e589-186d-4c13-a092-9ed53dea91aa service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:29.175257 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e589-186d-4c13-a092-9ed53dea91aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:29.175562 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e589-186d-4c13-a092-9ed53dea91aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:29.184501 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9e589-186d-4c13-a092-9ed53dea91aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:29.185936 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9e589-186d-4c13-a092-9ed53dea91aa service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:29.195182 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1449/2872] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:29.609956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:29.609956 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:29.619612 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855992fe-f302-448a-8e58-5c1a8798149f None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:29.629051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:29.629534 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:29.629823 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:29.630111 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] PATH_INFO: `/v3/projects/d1d04dbac9284e91911b5ad8fc1bbd5b` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:29.631327 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d04dbac9284e91911b5ad8fc1bbd5b)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:29.633259 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:29.735897 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d04dbac9284e91911b5ad8fc1bbd5b', 'request_id': 'req-089ab625-ff3d-45f8-ac06-53d5ff527624'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.809115 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d04dbac9284e91911b5ad8fc1bbd5b', 'request_id': 'req-089ab625-ff3d-45f8-ac06-53d5ff527624'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.814957 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d04dbac9284e91911b5ad8fc1bbd5b', 'request_id': 'req-089ab625-ff3d-45f8-ac06-53d5ff527624'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.826451 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] The token cache is being invalidate because project d1d04dbac9284e91911b5ad8fc1bbd5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:29.827700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.827700 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-089ab625-ff3d-45f8-ac06-53d5ff527624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:29.829840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1424/2873] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:29 2022] DELETE /identity/v3/projects/d1d04dbac9284e91911b5ad8fc1bbd5b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:29.835563 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:29.836092 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:29.836312 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:29.855051 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:29.856307 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:29.873735 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:29.917825 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1450/2874] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:29.925745 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:29.926054 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:29.926337 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:29.944205 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:29.945379 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:29.953223 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1425/2875] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:29.957676 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:29.968096 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d843e-12a5-4df6-bdc6-0ae5e9170a95 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:29.972943 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:29.973588 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:29.974509 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:29.974788 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:29.975542 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:47:29.975847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:29.977462 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:29.989392 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1451/2876] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:47:29.993558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.002741 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9adcab-51d9-45a7-b6d0-4cf05f3af46d None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.007359 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.008003 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.008284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.008559 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.009238 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.011113 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.084982 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1426/2877] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:47:29 2022] POST /identity/v3/projects => generated 359 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:30.089426 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.100201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a38b87-6c9c-40b0-b635-d7bc99b94a9e admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.155523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.156183 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.156466 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.156765 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.157440 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.159365 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.215391 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1452/2878] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:47:30 2022] POST /identity/v3/users => generated 280 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:47:30.219748 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.230141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ef2b-8df4-47a3-8b29-cd723cb60887 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.234721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.235399 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.235685 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.235961 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.236683 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.238263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.248654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1427/2879] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:30.252427 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.262653 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6158d5-597a-46c0-8d27-45578d5e1f34 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.267271 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80410386-242b-4acc-8044-223512712f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.267953 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80410386-242b-4acc-8044-223512712f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.268238 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80410386-242b-4acc-8044-223512712f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.268513 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80410386-242b-4acc-8044-223512712f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.269224 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80410386-242b-4acc-8044-223512712f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.271479 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80410386-242b-4acc-8044-223512712f87 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.281231 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1453/2880] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:30.285208 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.295141 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e845b6-4f64-40aa-91fd-2fc38fc1059b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.299806 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.300710 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.301318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.301318 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] PATH_INFO: `/v3/projects/941559e909934f19992b70aa3ea9d90b/users/4b354809e82a402da8441c7a342811a4/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.315909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] RBAC: Authorizing `identity:create_grant(project_id=941559e909934f19992b70aa3ea9d90b, user_id=4b354809e82a402da8441c7a342811a4, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.319058 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.387032 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1428/2881] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:30 2022] PUT /identity/v3/projects/941559e909934f19992b70aa3ea9d90b/users/4b354809e82a402da8441c7a342811a4/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:30.398864 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.399158 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.399497 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.425198 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] MFA Rules not processed for user `4b354809e82a402da8441c7a342811a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:30.426384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:30.489913 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1454/2882] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:30 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:30.520884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.522587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:30.544572 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:30.579821 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd02bc2-4d6f-4c99-bfcb-bb83bd4c410b admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.584524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a7a0f-684d-4c08-9b1b-ff61b6b58f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.585155 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a7a0f-684d-4c08-9b1b-ff61b6b58f91 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.589598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a7a0f-684d-4c08-9b1b-ff61b6b58f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.589598 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a7a0f-684d-4c08-9b1b-ff61b6b58f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.594065 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a7a0f-684d-4c08-9b1b-ff61b6b58f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.595515 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a7a0f-684d-4c08-9b1b-ff61b6b58f91 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.604704 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1429/2883] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:30 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:30.619322 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:30.639030 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:30.681912 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde42919-4309-4b1a-9072-698d627dcdfb None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:30.686560 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:30.687510 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.687801 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.688079 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] PATH_INFO: `/v3/users/4b354809e82a402da8441c7a342811a4` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.689279 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b354809e82a402da8441c7a342811a4)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:30.691191 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:30.872968 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b354809e82a402da8441c7a342811a4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:30.873650 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b354809e82a402da8441c7a342811a4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:30.885389 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b354809e82a402da8441c7a342811a4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:30.893042 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b354809e82a402da8441c7a342811a4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:30.956670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b354809e82a402da8441c7a342811a4'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:30.962955 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1455/2884] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:30 2022] DELETE /identity/v3/users/4b354809e82a402da8441c7a342811a4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:30.967604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:30.971175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:30.971175 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:30.986309 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:30.987524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:31.046575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1430/2885] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:31.056852 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:31.058503 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:31.116064 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fe867-c217-4178-810c-0a355c50ec2f admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:31.119675 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72f876-6a69-4c5c-b3d4-689f8d0057fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:31.120360 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72f876-6a69-4c5c-b3d4-689f8d0057fd service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:31.120644 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72f876-6a69-4c5c-b3d4-689f8d0057fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:31.120931 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72f876-6a69-4c5c-b3d4-689f8d0057fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:31.129843 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72f876-6a69-4c5c-b3d4-689f8d0057fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:31.131213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72f876-6a69-4c5c-b3d4-689f8d0057fd service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:31.140657 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1456/2886] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:31.553935 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:31.555516 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:31.567634 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e76d1c-3943-4767-80ee-482c73f4f86e None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:31.572333 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:31.573221 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:31.573585 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:31.573875 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] PATH_INFO: `/v3/projects/941559e909934f19992b70aa3ea9d90b` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:31.575231 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=941559e909934f19992b70aa3ea9d90b)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:31.577162 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:31.701680 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941559e909934f19992b70aa3ea9d90b', 'request_id': 'req-f5597124-4819-435e-8983-14f24d2eac18'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:31.763752 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941559e909934f19992b70aa3ea9d90b', 'request_id': 'req-f5597124-4819-435e-8983-14f24d2eac18'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:31.771281 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941559e909934f19992b70aa3ea9d90b', 'request_id': 'req-f5597124-4819-435e-8983-14f24d2eac18'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:31.782596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] The token cache is being invalidate because project 941559e909934f19992b70aa3ea9d90b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:31.784678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:31.784678 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-f5597124-4819-435e-8983-14f24d2eac18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:31.785048 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1431/2887] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:31 2022] DELETE /identity/v3/projects/941559e909934f19992b70aa3ea9d90b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:31.789847 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:31.790140 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:31.790448 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:31.807679 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:31.808922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:31.843437 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:31.874374 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1457/2888] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:31.882868 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:31.883164 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:31.883491 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:31.902460 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:31.903590 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:31.911423 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1432/2889] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:31.915686 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:31.925108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f5f7c-3fd0-40f7-8489-015ca7f49473 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:31.930384 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:31.931038 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:31.931351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:31.931634 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:31.932357 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:47:31.932664 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:31.934294 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:31.946302 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1458/2890] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:47:31.950564 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:31.960396 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca076-d618-442d-88ff-190356dd8cbb None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:31.965010 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:31.965655 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:31.965940 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:31.966215 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:31.966898 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:31.968963 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.050415 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1433/2891] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:47:31 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:32.055314 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:32.065256 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93099ed-542a-462e-8b36-2ba7834a7bf5 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:32.120001 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:32.120670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.122790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.122790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.122790 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:32.123967 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.242893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1459/2892] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:47:32 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:32.247416 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:32.258255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18677371-ebef-4b06-bd28-ce185f33ed16 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:32.262884 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:32.263575 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.263862 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.264139 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.264863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:32.266454 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.278596 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1434/2893] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:47:32.282606 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:32.292520 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e6a3d-85fe-4598-a99e-c8a9c3e60d40 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:32.297219 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b1f9-5223-4b17-a8c2-b9270591ad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:32.297876 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b1f9-5223-4b17-a8c2-b9270591ad1b admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.298162 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b1f9-5223-4b17-a8c2-b9270591ad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.298439 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b1f9-5223-4b17-a8c2-b9270591ad1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.299165 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b1f9-5223-4b17-a8c2-b9270591ad1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:32.300817 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b1f9-5223-4b17-a8c2-b9270591ad1b admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.311543 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1460/2894] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 01 04:47:32.315587 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:32.326035 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9c8a3-9b31-48a2-b6b9-2d0dbc9e2455 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:32.330560 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:32.331464 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.331749 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.332023 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] PATH_INFO: `/v3/projects/8769e3ab42414a83aa3d6ed4f5fd1e6e/users/438ebff98a4745f7ae79d751ac73907b/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.334237 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:32.347120 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8769e3ab42414a83aa3d6ed4f5fd1e6e, user_id=438ebff98a4745f7ae79d751ac73907b, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:32.350604 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.425871 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1435/2895] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:32 2022] PUT /identity/v3/projects/8769e3ab42414a83aa3d6ed4f5fd1e6e/users/438ebff98a4745f7ae79d751ac73907b/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:32.436717 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819884a-1821-46c8-b003-58372f065524 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.437009 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819884a-1821-46c8-b003-58372f065524 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.437291 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819884a-1821-46c8-b003-58372f065524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.463327 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-c819884a-1821-46c8-b003-58372f065524 None None] MFA Rules not processed for user `438ebff98a4745f7ae79d751ac73907b`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:32.464861 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-c819884a-1821-46c8-b003-58372f065524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:32.528054 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1461/2896] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:32.539787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:32.541509 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:32.597210 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69b157-b510-41a4-bce9-a4967f8ff84a admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:32.601332 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11267183-f2c6-48e1-bc87-3ba07e5887a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:32.601974 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11267183-f2c6-48e1-bc87-3ba07e5887a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.602255 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11267183-f2c6-48e1-bc87-3ba07e5887a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.602531 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11267183-f2c6-48e1-bc87-3ba07e5887a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.611146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11267183-f2c6-48e1-bc87-3ba07e5887a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:32.612651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11267183-f2c6-48e1-bc87-3ba07e5887a0 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.622051 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1436/2897] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:32.646113 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819884a-1821-46c8-b003-58372f065524 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:32.685523 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:32.708471 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819884a-1821-46c8-b003-58372f065524 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:32.713250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:32.714111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:32.714394 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:32.714670 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] PATH_INFO: `/v3/users/438ebff98a4745f7ae79d751ac73907b` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:32.715893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438ebff98a4745f7ae79d751ac73907b)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:32.717954 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:32.884713 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438ebff98a4745f7ae79d751ac73907b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:32.885351 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438ebff98a4745f7ae79d751ac73907b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:32.897727 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438ebff98a4745f7ae79d751ac73907b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:32.904662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438ebff98a4745f7ae79d751ac73907b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.019430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438ebff98a4745f7ae79d751ac73907b'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.026287 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1462/2898] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:32 2022] DELETE /identity/v3/users/438ebff98a4745f7ae79d751ac73907b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:33.030924 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.031246 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.031558 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.049482 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:33.050662 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:33.107903 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1437/2899] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:33.117757 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:33.119125 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:33.176016 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fa046-81e2-4c9a-b5b7-17826c8f9cf7 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:33.179455 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88feda87-786f-41cd-8e35-dba0eb4f1ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:33.180094 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feda87-786f-41cd-8e35-dba0eb4f1ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.180375 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feda87-786f-41cd-8e35-dba0eb4f1ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.180647 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feda87-786f-41cd-8e35-dba0eb4f1ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.189362 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feda87-786f-41cd-8e35-dba0eb4f1ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:33.190812 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feda87-786f-41cd-8e35-dba0eb4f1ddc service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:33.200681 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1463/2900] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:33.571282 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:33.574840 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:33.585132 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ea2e7-0dc3-4a2e-a337-6e9ad9a1b140 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:33.589863 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:33.590721 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.591002 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.591284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] PATH_INFO: `/v3/projects/8769e3ab42414a83aa3d6ed4f5fd1e6e` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.592527 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8769e3ab42414a83aa3d6ed4f5fd1e6e)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:33.594473 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:33.712057 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8769e3ab42414a83aa3d6ed4f5fd1e6e', 'request_id': 'req-9a6caaf3-ab4b-40cc-96ce-6067a3868063'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.785163 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8769e3ab42414a83aa3d6ed4f5fd1e6e', 'request_id': 'req-9a6caaf3-ab4b-40cc-96ce-6067a3868063'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.791128 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8769e3ab42414a83aa3d6ed4f5fd1e6e', 'request_id': 'req-9a6caaf3-ab4b-40cc-96ce-6067a3868063'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.802757 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] The token cache is being invalidate because project 8769e3ab42414a83aa3d6ed4f5fd1e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:33.803047 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.803702 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-9a6caaf3-ab4b-40cc-96ce-6067a3868063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:33.805329 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1438/2901] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:33 2022] DELETE /identity/v3/projects/8769e3ab42414a83aa3d6ed4f5fd1e6e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:33.810527 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.810820 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.811103 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.828906 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:33.830111 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:33.880577 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:33.890250 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1464/2902] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:33.899112 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.899436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.899722 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.919031 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:33.920263 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:33.928993 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1439/2903] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:33.933146 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:33.942805 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e47d5-34e2-4579-a9ad-e0280b56de52 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:33.947950 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:33.949893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.949893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.949893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.949893 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80710) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 01 04:47:33.950304 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:33.951837 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:33.963922 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1465/2904] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jun 1 04:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 01 04:47:33.968236 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:33.972883 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:33.978870 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9c455-151c-40d0-9591-1463e784dc2a None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:33.983665 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:33.984320 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:33.984609 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:33.984891 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] PATH_INFO: `/v3/projects` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:33.985619 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:33.987536 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:34.125207 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1440/2905] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jun 1 04:47:33 2022] POST /identity/v3/projects => generated 397 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 01 04:47:34.129454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:34.139268 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edfaab-1515-40e8-805b-66d614e08bd5 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:34.193660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:34.194321 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:34.194602 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:34.194879 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] PATH_INFO: `/v3/users` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:34.195583 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:34.197526 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:34.272618 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1466/2906] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jun 1 04:47:34 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 01 04:47:34.276607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:34.288093 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29544d-bfbb-47a2-959f-aefaad0e644e admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:34.292706 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:34.293366 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:34.293654 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:34.293932 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:34.294681 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:34.296310 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:34.306085 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1441/2907] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:34.310239 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:34.320189 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6172b8-a9bb-4385-abf3-8c2dccd665af admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:34.324880 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba231d-9ba1-4648-8ca1-0714f9898398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:34.325533 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba231d-9ba1-4648-8ca1-0714f9898398 admin admin] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:34.325814 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba231d-9ba1-4648-8ca1-0714f9898398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:34.326090 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba231d-9ba1-4648-8ca1-0714f9898398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:34.326811 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba231d-9ba1-4648-8ca1-0714f9898398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:34.328438 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba231d-9ba1-4648-8ca1-0714f9898398 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:34.338739 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1467/2908] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jun 1 04:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 01 04:47:34.342998 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:34.353756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760511e3-560b-4c74-ab9d-3c94af6a8987 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:34.358252 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:34.359121 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:34.359436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:34.359717 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] PATH_INFO: `/v3/projects/233fe5bccb054ef887d9ce25e090ce2b/users/6a3fbad089e14708a57072182d9b5400/roles/b79d035ffc764400a134a4adcda8b7f8` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:34.374264 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233fe5bccb054ef887d9ce25e090ce2b, user_id=6a3fbad089e14708a57072182d9b5400, role_id=b79d035ffc764400a134a4adcda8b7f8)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:34.376917 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:34.466812 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1442/2909] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jun 1 04:47:34 2022] PUT /identity/v3/projects/233fe5bccb054ef887d9ce25e090ce2b/users/6a3fbad089e14708a57072182d9b5400/roles/b79d035ffc764400a134a4adcda8b7f8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 01 04:47:34.477856 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] REQUEST_METHOD: `POST` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:34.478148 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:34.478430 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:34.504033 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.auth.core [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] MFA Rules not processed for user `6a3fbad089e14708a57072182d9b5400`. Rule list: `[]` (Enabled: `True`). {{(pid=80710) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:34.505213 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:34.568144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1468/2910] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:34.579159 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:34.580893 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:34.638407 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e158-fd0b-48e2-a0b5-c77c093e9480 admin admin] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:34.642070 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e305da-5f8f-442d-b0cd-e9a5feb79900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f4e0e17040d41bf91cf4022dc4b4d4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:34.642723 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e305da-5f8f-442d-b0cd-e9a5feb79900 service cinder] REQUEST_METHOD: `GET` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:34.643012 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e305da-5f8f-442d-b0cd-e9a5feb79900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:34.643295 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e305da-5f8f-442d-b0cd-e9a5feb79900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:34.652284 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e305da-5f8f-442d-b0cd-e9a5feb79900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:34.653756 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e305da-5f8f-442d-b0cd-e9a5feb79900 service cinder] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:34.662435 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1443/2911] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:43.529277 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:43.580595 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:43.591915 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4a4d-cc14-4fc5-acbf-15fe61567637 None None] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:43.596662 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:43.597530 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:43.597810 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:43.598132 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] PATH_INFO: `/v3/users/6a3fbad089e14708a57072182d9b5400` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:43.599868 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3fbad089e14708a57072182d9b5400)` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:43.601774 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:43.805672 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3fbad089e14708a57072182d9b5400'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:43.806454 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3fbad089e14708a57072182d9b5400'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:43.818144 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3fbad089e14708a57072182d9b5400'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:43.824962 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3fbad089e14708a57072182d9b5400'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:43.894771 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.notifications [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3fbad089e14708a57072182d9b5400'} {{(pid=80710) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:43.901660 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1469/2912] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jun 1 04:47:43 2022] DELETE /identity/v3/users/6a3fbad089e14708a57072182d9b5400 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:43.906170 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] REQUEST_METHOD: `POST` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:43.906651 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:43.906787 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:43.924146 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.auth.core [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] MFA Rules not processed for user `c2efe34f830a400eb5eb8876be2a1da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 01 04:47:43.925328 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:43.984248 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1444/2913] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jun 1 04:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 01 04:47:43.994201 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:43.995764 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:44.051899 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46d203-96bf-485e-84f6-d12e03f0dd52 admin admin] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:44.056018 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '885cc3a3c23c451c9ad7526e7fa83639', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:44.056658 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:44.056940 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:44.057217 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:44.066313 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:44.067785 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:44.077756 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1470/2914] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:47:44.460614 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] Authenticating user token {{(pid=80711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:47:44.462045 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.fernet_utils [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:47:44.474243 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1f18e-b1cc-4698-9df1-b80b64fc8149 None None] Validating token access rules against request {{(pid=80711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:47:44.479636 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2efe34f830a400eb5eb8876be2a1da6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efd554ba71f4463abb4c78dc3cafe52a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:47:44.480524 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:47:44.480955 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:47:44.481241 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] PATH_INFO: `/v3/projects/233fe5bccb054ef887d9ce25e090ce2b` {{(pid=80711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:47:44.482436 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233fe5bccb054ef887d9ce25e090ce2b)` {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:47:44.484607 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] RBAC: Authorization granted {{(pid=80711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:47:44.576882 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:44.580526 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233fe5bccb054ef887d9ce25e090ce2b', 'request_id': 'req-8428555d-2e92-4e48-9b34-2c5b7957e8d3'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:44.660421 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233fe5bccb054ef887d9ce25e090ce2b', 'request_id': 'req-8428555d-2e92-4e48-9b34-2c5b7957e8d3'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:44.666474 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233fe5bccb054ef887d9ce25e090ce2b', 'request_id': 'req-8428555d-2e92-4e48-9b34-2c5b7957e8d3'} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:44.677610 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] The token cache is being invalidate because project 233fe5bccb054ef887d9ce25e090ce2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80711) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 01 04:47:44.677909 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:44.679081 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG keystone.notifications [None req-8428555d-2e92-4e48-9b34-2c5b7957e8d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80711) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 01 04:47:44.680708 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: [pid: 80711|app: 0|req: 1445/2915] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jun 1 04:47:44 2022] DELETE /identity/v3/projects/233fe5bccb054ef887d9ce25e090ce2b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 01 04:47:54.197052 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:47:54.717766 dv-cinder-667108-37-iscsi devstack@keystone.service[80711]: DEBUG dbcounter [-] [80711] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80711) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 01 04:51:52.740303 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] Authenticating user token {{(pid=80710) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 01 04:51:52.741755 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:51:52.883619 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f824c8-86c2-428b-a5f4-cc59edf9a75e service neutron] Validating token access rules against request {{(pid=80710) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 01 04:51:52.946802 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6848ac67d15b406693e33c9dfabbbe89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c694617dec334b69a78a72cf36fe63dc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80710) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 01 04:51:52.947546 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] REQUEST_METHOD: `GET` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 01 04:51:52.947831 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] SCRIPT_NAME: `/identity` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 01 04:51:52.948108 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80710) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 01 04:51:52.949863 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.fernet_utils [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80710) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 01 04:51:52.961666 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 01 04:51:52.963225 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508871c1-eb94-465a-9e95-6c9d531bf334 service placement] RBAC: Authorization granted {{(pid=80710) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 01 04:51:52.973958 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: [pid: 80710|app: 0|req: 1471/2916] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jun 1 04:51:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 01 04:52:02.969159 dv-cinder-667108-37-iscsi devstack@keystone.service[80710]: DEBUG dbcounter [-] [80710] Writing DB stats keystone:SELECT=89 {{(pid=80710) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}